[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:37:15 dist-test-slave-07c2 krb5kdc[14408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:37:15 dist-test-slave-07c2 krb5kdc[14408](info): set up 2 sockets
Aug 07 19:37:15 dist-test-slave-07c2 krb5kdc[14408](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 19:37:16.788669 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.003s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:37:16 dist-test-slave-07c2 krb5kdc[14408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:37:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:37:16.979516 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:38951
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:40411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:38951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:17.307571 14421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:17.307936 14421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:17.308104 14421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:17.315357 14421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:17.315506 14421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:17.315618 14421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:17.315723 14421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:17.315812 14421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:17.325625 14421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:40411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:38951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:38951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:17.327208 14421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:17.328555 14421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:17.340870 14427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:17.340829 14431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:17.340922 14428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:17.341395 14421 server_base.cc:1034] running on GCE node
I20260807 19:37:17.342166 14421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:17.343474 14421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:17.344681 14421 hybrid_clock.cc:648] HybridClock initialized: now 1786131437344638 us; error 76 us; skew 500 ppm
I20260807 19:37:17.345036 14421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:17 dist-test-slave-07c2 krb5kdc[14408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131437, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:17.348094 14421 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:37:17.349457 14421 webserver.cc:460] Webserver started at http://127.14.16.126:38567/ using document root <none> and password file <none>
I20260807 19:37:17.350078 14421 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:17.350175 14421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:17.350418 14421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:17.353276 14421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "8da364f8b727461886578179a8ed9a8c"
format_stamp: "Formatted at 2026-08-07 19:37:17 on dist-test-slave-07c2"
server_key: "965b28045e0e4fb27d45b0b80cbf148a"
server_key_iv: "c18f3d2f5a8a83228017e5b64c7f0e71"
server_key_version: "encryptionkey@0"
I20260807 19:37:17.353963 14421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "8da364f8b727461886578179a8ed9a8c"
format_stamp: "Formatted at 2026-08-07 19:37:17 on dist-test-slave-07c2"
server_key: "965b28045e0e4fb27d45b0b80cbf148a"
server_key_iv: "c18f3d2f5a8a83228017e5b64c7f0e71"
server_key_version: "encryptionkey@0"
I20260807 19:37:17.358759 14421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 19:37:17.361977 14438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:17.363188 14421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:37:17.363394 14421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "8da364f8b727461886578179a8ed9a8c"
format_stamp: "Formatted at 2026-08-07 19:37:17 on dist-test-slave-07c2"
server_key: "965b28045e0e4fb27d45b0b80cbf148a"
server_key_iv: "c18f3d2f5a8a83228017e5b64c7f0e71"
server_key_version: "encryptionkey@0"
I20260807 19:37:17.363586 14421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:17.377079 14421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:17.380999 14421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:17.381281 14421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:17.400525 14421 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:38951
I20260807 19:37:17.400527 14489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:38951 every 8 connection(s)
I20260807 19:37:17.401959 14421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:17.405620 14491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:17.406646 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 14421
I20260807 19:37:17.406960 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:17.408870 14401 external_mini_cluster.cc:1428] Setting key bc71022e74246598576f9a9226953ea0
I20260807 19:37:17.416770 14491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c: Bootstrap starting.
I20260807 19:37:17.419615 14491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:17.420620 14491 log.cc:824] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:17.423502 14491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c: No bootstrap required, opened a new log
I20260807 19:37:17.430645 14491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8da364f8b727461886578179a8ed9a8c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38951 } }
I20260807 19:37:17.431041 14491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:17.431164 14491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da364f8b727461886578179a8ed9a8c, State: Initialized, Role: FOLLOWER
I20260807 19:37:17.431900 14491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8da364f8b727461886578179a8ed9a8c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38951 } }
I20260807 19:37:17.432226 14491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:17.432391 14491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:17.432578 14491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:17.436019 14491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8da364f8b727461886578179a8ed9a8c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38951 } }
I20260807 19:37:17.436618 14491 leader_election.cc:302] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8da364f8b727461886578179a8ed9a8c; no voters:
Aug 07 19:37:17 dist-test-slave-07c2 krb5kdc[14408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:17.437307 14491 leader_election.cc:288] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:17.437428 14496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:17.439486 14491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:17.440099 14496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [term 1 LEADER]: Becoming Leader. State: Replica: 8da364f8b727461886578179a8ed9a8c, State: Running, Role: LEADER
I20260807 19:37:17.440789 14496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8da364f8b727461886578179a8ed9a8c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38951 } }
I20260807 19:37:17.443894 14499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8da364f8b727461886578179a8ed9a8c. Latest consensus state: current_term: 1 leader_uuid: "8da364f8b727461886578179a8ed9a8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8da364f8b727461886578179a8ed9a8c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38951 } } }
I20260807 19:37:17.444285 14499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:17.444834 14497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8da364f8b727461886578179a8ed9a8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8da364f8b727461886578179a8ed9a8c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38951 } } }
I20260807 19:37:17.445158 14497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:17.447394 14505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:17.451730 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:17.412577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.14.16.126:38951)
0807 19:37:17.413464 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:17.413482 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:17.414589 (+ 1107us) server_negotiation.cc:407] Connection header received
0807 19:37:17.423527 (+ 8938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:17.423571 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:17.423951 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:17.424337 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:17.427145 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:17.428569 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:17.430235 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:17.430908 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:17.431155 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:17.438624 (+ 7469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:17.438639 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:17.438665 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:17.438735 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:17.444217 (+ 5482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:17.445931 (+ 1714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:17.445943 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:17.445955 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:17.446233 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:17.446990 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:17.446998 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:17.447004 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:17.448176 (+ 1172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:17.448516 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:17.449743 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0807 19:37:17.450172 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":417,"threads_started":1}
I20260807 19:37:17.454774 14505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:17.464869 14516 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:37:17.466252 14505 catalog_manager.cc:1347] Generated new cluster ID: 7626fd10f93e4ae58df9227dff1e05ba
I20260807 19:37:17.466360 14505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:17.484429 14505 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:17.485785 14505 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:17.493698 14505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8da364f8b727461886578179a8ed9a8c: Generated new TSK 0
I20260807 19:37:17.494637 14505 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:37:17.549079 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:38951
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:40411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:17.885458 14519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:17.885815 14519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:17.885951 14519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:17.893182 14519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:17.893323 14519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:17.893535 14519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:37:17.903429 14519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:40411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:38951
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:17.904764 14519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:17.906211 14519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:17.920145 14525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:17.920114 14527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:17.920190 14524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:17.927783 14519 server_base.cc:1034] running on GCE node
I20260807 19:37:17.928406 14519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:17.929399 14519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:17.930603 14519 hybrid_clock.cc:648] HybridClock initialized: now 1786131437930563 us; error 36 us; skew 500 ppm
I20260807 19:37:17.930912 14519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:17 dist-test-slave-07c2 krb5kdc[14408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131437, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:17.934175 14519 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:37:17.935875 14519 webserver.cc:460] Webserver started at http://127.14.16.65:46021/ using document root <none> and password file <none>
I20260807 19:37:17.936635 14519 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:17.936782 14519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:17.937103 14519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:17.940055 14519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "1c453a494f4a4bbca862aebb2e2739e6"
format_stamp: "Formatted at 2026-08-07 19:37:17 on dist-test-slave-07c2"
server_key: "3223518e513032d51510838e52e13f5f"
server_key_iv: "4983530e0dcd536fd827b99a507795e5"
server_key_version: "encryptionkey@0"
I20260807 19:37:17.940810 14519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "1c453a494f4a4bbca862aebb2e2739e6"
format_stamp: "Formatted at 2026-08-07 19:37:17 on dist-test-slave-07c2"
server_key: "3223518e513032d51510838e52e13f5f"
server_key_iv: "4983530e0dcd536fd827b99a507795e5"
server_key_version: "encryptionkey@0"
I20260807 19:37:17.945782 14519 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:37:17.949735 14534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:17.951038 14519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 19:37:17.951213 14519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "1c453a494f4a4bbca862aebb2e2739e6"
format_stamp: "Formatted at 2026-08-07 19:37:17 on dist-test-slave-07c2"
server_key: "3223518e513032d51510838e52e13f5f"
server_key_iv: "4983530e0dcd536fd827b99a507795e5"
server_key_version: "encryptionkey@0"
I20260807 19:37:17.951521 14519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:17.974313 14519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:17.978888 14519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:17.979199 14519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:17.980150 14519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:17.981572 14519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:17.981674 14519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:17.981837 14519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:17.981910 14519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:18.017477 14519 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:38849
I20260807 19:37:18.017573 14649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:38849 every 8 connection(s)
I20260807 19:37:18.018910 14519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:37:18.028378 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 14519
I20260807 19:37:18.028640 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:37:18.029309 14401 external_mini_cluster.cc:1428] Setting key 18097ba47b1a18ff3f3aa9a478cb1575
Aug 07 19:37:18 dist-test-slave-07c2 krb5kdc[14408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131437, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:18.041338 14652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:18.021581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:56725 (local address 127.14.16.126:38951)
0807 19:37:18.022036 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:18.022041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:18.023038 (+ 997us) server_negotiation.cc:407] Connection header received
0807 19:37:18.024992 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:18.024999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:18.025099 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:18.025221 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:18.026685 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:18.027492 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:18.029212 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:18.029672 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:18.029822 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:18.033842 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:18.033851 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:18.033859 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:18.033908 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:18.037542 (+ 3634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:18.038564 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:18.038570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:18.038573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:18.038713 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:18.039246 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:18.039250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:18.039252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:18.039677 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:18.039799 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:18.040812 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0807 19:37:18.041039 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":230,"threads_started":1}
I20260807 19:37:18.042865 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:18.022148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:38951 (local address 127.14.16.65:56725)
0807 19:37:18.022877 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:18.022914 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:18.024723 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:18.025453 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:18.025461 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:18.025958 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:18.026517 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:18.026533 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:18.027663 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:18.027668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:18.029061 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:18.029074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:18.029914 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:18.029922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:18.030248 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:18.030999 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:18.031021 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:18.033605 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:18.037690 (+ 4085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:18.037697 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:18.037711 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:18.038420 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:18.038823 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:18.038829 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:18.038834 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:18.039120 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:18.039821 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:18.039832 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:18.040315 (+ 483us) client_negotiation.cc:769] Sending connection context
0807 19:37:18.040652 (+ 337us) client_negotiation.cc:240] Negotiation successful
0807 19:37:18.041290 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":286,"threads_started":1}
I20260807 19:37:18.044662 14650 heartbeater.cc:347] Connected to a master server at 127.14.16.126:38951
I20260807 19:37:18.045024 14650 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:18.045881 14650 heartbeater.cc:511] Master 127.14.16.126:38951 requested a full tablet report, sending...
I20260807 19:37:18.048305 14455 ts_manager.cc:194] Registered new tserver with Master: 1c453a494f4a4bbca862aebb2e2739e6 (127.14.16.65:38849)
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
I20260807 19:37:18.049861 14455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:56725
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:37:18.075001 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:38951
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:40411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:18.399367 14656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:18.399752 14656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:18.399969 14656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:18.407533 14656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:18.407681 14656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:18.407873 14656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:37:18.417914 14656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:40411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:38951
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:18.419220 14656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:18.420661 14656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:18.433910 14663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:18.433998 14662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:18.433957 14665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:18.434741 14656 server_base.cc:1034] running on GCE node
I20260807 19:37:18.435487 14656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:18.436404 14656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:18.437631 14656 hybrid_clock.cc:648] HybridClock initialized: now 1786131438437590 us; error 55 us; skew 500 ppm
I20260807 19:37:18.437945 14656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:18 dist-test-slave-07c2 krb5kdc[14408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131438, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:18.441311 14656 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:37:18.442629 14656 webserver.cc:460] Webserver started at http://127.14.16.66:38577/ using document root <none> and password file <none>
I20260807 19:37:18.443245 14656 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:18.443408 14656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:18.443674 14656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:18.446790 14656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "bd9aea100dd44d4ab6c5b7b13ce22363"
format_stamp: "Formatted at 2026-08-07 19:37:18 on dist-test-slave-07c2"
server_key: "634b5655cfb0278d5d10ef4a139cf753"
server_key_iv: "fc046c1ecc54607685c6018acba88bcb"
server_key_version: "encryptionkey@0"
I20260807 19:37:18.447642 14656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "bd9aea100dd44d4ab6c5b7b13ce22363"
format_stamp: "Formatted at 2026-08-07 19:37:18 on dist-test-slave-07c2"
server_key: "634b5655cfb0278d5d10ef4a139cf753"
server_key_iv: "fc046c1ecc54607685c6018acba88bcb"
server_key_version: "encryptionkey@0"
I20260807 19:37:18.452816 14656 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 19:37:18.456307 14672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:18.457696 14656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:37:18.457834 14656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "bd9aea100dd44d4ab6c5b7b13ce22363"
format_stamp: "Formatted at 2026-08-07 19:37:18 on dist-test-slave-07c2"
server_key: "634b5655cfb0278d5d10ef4a139cf753"
server_key_iv: "fc046c1ecc54607685c6018acba88bcb"
server_key_version: "encryptionkey@0"
I20260807 19:37:18.457995 14656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:18.476495 14656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:18.480338 14656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:18.480578 14656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:18.481511 14656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:18.482909 14656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:18.482985 14656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:18.483072 14656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:18.483116 14656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:18.517735 14656 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:43347
I20260807 19:37:18.517791 14786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:43347 every 8 connection(s)
I20260807 19:37:18.519062 14656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:37:18.522977 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 14656
I20260807 19:37:18.523178 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:37:18.523689 14401 external_mini_cluster.cc:1428] Setting key 49617c7fe59a0da7773ac56039b6dd79
Aug 07 19:37:18 dist-test-slave-07c2 krb5kdc[14408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131438, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:18.539883 14652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:18.521791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:48183 (local address 127.14.16.126:38951)
0807 19:37:18.521978 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:18.521982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:18.523126 (+ 1144us) server_negotiation.cc:407] Connection header received
0807 19:37:18.525498 (+ 2372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:18.525506 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:18.525599 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:18.525677 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:18.527138 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:18.527928 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:18.529592 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:18.530096 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:18.530234 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:18.533771 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:18.533776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:18.533780 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:18.533811 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:18.536407 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:18.537414 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:18.537422 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:18.537429 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:18.537610 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:18.538077 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:18.538084 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:18.538091 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:18.538367 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:18.538529 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:18.539253 (+ 724us) server_negotiation.cc:299] Negotiation successful
0807 19:37:18.539516 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:37:18.540946 14789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:18.522254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:38951 (local address 127.14.16.66:48183)
0807 19:37:18.522952 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:18.522996 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:18.525162 (+ 2166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:18.525950 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:18.525958 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:18.526448 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:18.526975 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:18.526991 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:18.528092 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:18.528096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:18.529322 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:18.529334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:18.530339 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:18.530347 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:18.530573 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:18.531275 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:18.531360 (+ 85us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:18.533544 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:18.536708 (+ 3164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:18.536737 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:18.536753 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:18.537250 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:18.537730 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:18.537736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:18.537740 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:18.537949 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:18.538498 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:18.538506 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:18.538874 (+ 368us) client_negotiation.cc:769] Sending connection context
0807 19:37:18.539175 (+ 301us) client_negotiation.cc:240] Negotiation successful
0807 19:37:18.539774 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":298,"threads_started":1}
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
I20260807 19:37:18.542572 14787 heartbeater.cc:347] Connected to a master server at 127.14.16.126:38951
I20260807 19:37:18.542900 14787 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:18.543735 14787 heartbeater.cc:511] Master 127.14.16.126:38951 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
I20260807 19:37:18.545377 14455 ts_manager.cc:194] Registered new tserver with Master: bd9aea100dd44d4ab6c5b7b13ce22363 (127.14.16.66:43347)
I20260807 19:37:18.546201 14455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:48183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:37:18.568073 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:38951
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:40411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:18.897733 14792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:18.898003 14792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:18.898101 14792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:18.905524 14792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:18.905639 14792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:18.905755 14792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:37:18.915616 14792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:40411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:38951
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:18.916848 14792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:18.918121 14792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:18.930702 14799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:18.930788 14798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:18.930756 14801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:18.931834 14792 server_base.cc:1034] running on GCE node
I20260807 19:37:18.932483 14792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:18.933350 14792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:18.934574 14792 hybrid_clock.cc:648] HybridClock initialized: now 1786131438934536 us; error 49 us; skew 500 ppm
I20260807 19:37:18.934888 14792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:18 dist-test-slave-07c2 krb5kdc[14408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131438, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:18.938038 14792 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:37:18.939528 14792 webserver.cc:460] Webserver started at http://127.14.16.67:45209/ using document root <none> and password file <none>
I20260807 19:37:18.940261 14792 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:18.940407 14792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:18.940712 14792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:18.943630 14792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "7ee8c6e48452491aa55fcb43f8894204"
format_stamp: "Formatted at 2026-08-07 19:37:18 on dist-test-slave-07c2"
server_key: "4571d501ea96dd32c6d1775011e12ab0"
server_key_iv: "51c1235986efb663c8d3f41904d4c894"
server_key_version: "encryptionkey@0"
I20260807 19:37:18.944384 14792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "7ee8c6e48452491aa55fcb43f8894204"
format_stamp: "Formatted at 2026-08-07 19:37:18 on dist-test-slave-07c2"
server_key: "4571d501ea96dd32c6d1775011e12ab0"
server_key_iv: "51c1235986efb663c8d3f41904d4c894"
server_key_version: "encryptionkey@0"
I20260807 19:37:18.949333 14792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:37:18.952646 14808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:18.953923 14792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:37:18.954097 14792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "7ee8c6e48452491aa55fcb43f8894204"
format_stamp: "Formatted at 2026-08-07 19:37:18 on dist-test-slave-07c2"
server_key: "4571d501ea96dd32c6d1775011e12ab0"
server_key_iv: "51c1235986efb663c8d3f41904d4c894"
server_key_version: "encryptionkey@0"
I20260807 19:37:18.954298 14792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:18.968829 14792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:18.972749 14792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:18.973061 14792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:18.974013 14792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:18.975421 14792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:18.975521 14792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:18.975674 14792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:18.975754 14792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:19.010788 14792 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:36659
I20260807 19:37:19.010833 14928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:36659 every 8 connection(s)
I20260807 19:37:19.012153 14792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:37:19.015772 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 14792
I20260807 19:37:19.015971 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:37:19.016517 14401 external_mini_cluster.cc:1428] Setting key 6f5bff2bc0bcf718ecfb5d7a3bcb009a
Aug 07 19:37:19 dist-test-slave-07c2 krb5kdc[14408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131438, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:19.035120 14652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:19.014760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:43261 (local address 127.14.16.126:38951)
0807 19:37:19.014948 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:19.014953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:19.016229 (+ 1276us) server_negotiation.cc:407] Connection header received
0807 19:37:19.018575 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:19.018581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:19.018709 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:19.018832 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:19.020514 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.021442 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.022949 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.023437 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.023569 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:19.027823 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:19.027829 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:19.027834 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:19.027877 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:19.031651 (+ 3774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:19.032671 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:19.032676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:19.032679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:19.032843 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:19.033316 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:19.033321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:19.033325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:19.033623 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:19.033795 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:19.034495 (+ 700us) server_negotiation.cc:299] Negotiation successful
0807 19:37:19.034717 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:37:19.036051 14934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:19.015170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:38951 (local address 127.14.16.67:43261)
0807 19:37:19.016049 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:19.016115 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:19.018255 (+ 2140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:19.019097 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:19.019108 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:19.019715 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:19.020353 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:19.020371 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.021594 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.021607 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:19.022807 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:19.022819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.023660 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.023668 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:19.023909 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:19.024514 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:19.024536 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:19.027617 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:19.031847 (+ 4230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:19.031859 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:19.031878 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:19.032508 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:19.032995 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:19.032999 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:19.033001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:19.033188 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:19.033757 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:19.033765 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:19.034112 (+ 347us) client_negotiation.cc:769] Sending connection context
0807 19:37:19.034386 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 19:37:19.034911 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":313,"threads_started":1}
I20260807 19:37:19.037448 14929 heartbeater.cc:347] Connected to a master server at 127.14.16.126:38951
I20260807 19:37:19.037783 14929 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:19.038506 14929 heartbeater.cc:511] Master 127.14.16.126:38951 requested a full tablet report, sending...
I20260807 19:37:19.040115 14455 ts_manager.cc:194] Registered new tserver with Master: 7ee8c6e48452491aa55fcb43f8894204 (127.14.16.67:36659)
I20260807 19:37:19.041375 14455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:43261
I20260807 19:37:19.050622 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:37:19.053267 14650 heartbeater.cc:503] Master 127.14.16.126:38951 was elected leader, sending a full tablet report...
Aug 07 19:37:19 dist-test-slave-07c2 krb5kdc[14408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131439, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:37:19 dist-test-slave-07c2 krb5kdc[14408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131439, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:37:19.107261 14939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:19.088941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48264 (local address 127.14.16.65:38849)
0807 19:37:19.090002 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:19.090013 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:19.090050 (+ 37us) server_negotiation.cc:407] Connection header received
0807 19:37:19.090138 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:19.090149 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:19.090457 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:19.090660 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:19.092120 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.093901 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.095123 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.096147 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.096297 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:19.100173 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:19.100190 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:19.100218 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:19.100264 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:19.104292 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:19.104908 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:19.104918 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:19.104922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:19.105091 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:19.105474 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:19.105481 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:19.105486 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:19.105957 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:19.106306 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:19.106479 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 19:37:19.106758 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":861,"thread_start_us":683,"threads_started":1}
W20260807 19:37:19.108724 14562 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48264
Aug 07 19:37:19 dist-test-slave-07c2 krb5kdc[14408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:37:19 dist-test-slave-07c2 krb5kdc[14408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:37:19.159103 14939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:19.144933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.14.16.65:38849)
0807 19:37:19.145175 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:19.145184 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:19.145505 (+ 321us) server_negotiation.cc:407] Connection header received
0807 19:37:19.145730 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:19.145734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:19.145852 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:19.145963 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:19.147335 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.148103 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.149424 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.149852 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.149987 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:19.153871 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:19.153875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:19.153879 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:19.153913 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:19.156746 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:19.157347 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:19.157351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:19.157353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:19.157466 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:19.157840 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:19.157861 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:19.157866 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:19.158164 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:19.158351 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:19.158609 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 19:37:19.158806 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:37:19.160652 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 14519
I20260807 19:37:19.173524 14939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:19.161643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48280 (local address 127.14.16.65:38849)
0807 19:37:19.161924 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:19.161930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:19.162227 (+ 297us) server_negotiation.cc:407] Connection header received
0807 19:37:19.162394 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:19.162400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:19.162464 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:19.162563 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:19.163817 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.164503 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.165760 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.166191 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.166331 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:19.168606 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:19.168610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:19.168613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:19.168642 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:19.170955 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:19.171641 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:19.171648 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:19.171651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:19.171788 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:19.172153 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:19.172158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:19.172161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:19.172419 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:19.172671 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:19.172966 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 19:37:19.173173 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:37:19.174201 14645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:19.257813 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 14519
I20260807 19:37:19.276964 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 14656
Aug 07 19:37:19 dist-test-slave-07c2 krb5kdc[14408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:37:19.297025 14946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:19.277866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37198 (local address 127.14.16.66:43347)
0807 19:37:19.278411 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:19.278418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:19.278441 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:37:19.278499 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:19.278506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:19.278706 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:19.278884 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:19.280193 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.281210 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.282513 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.283085 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.283238 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:19.291254 (+ 8016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:19.291270 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:19.291329 (+ 59us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:19.291398 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:19.294412 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:19.295052 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:19.295058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:19.295061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:19.295190 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:19.295633 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:19.295636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:19.295638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:19.295999 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:19.296219 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:19.296520 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 19:37:19.296726 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":275,"threads_started":1}
I20260807 19:37:19.298326 14782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:19.381880 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 14656
I20260807 19:37:19.399894 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 14792
Aug 07 19:37:19 dist-test-slave-07c2 krb5kdc[14408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:37:19.416742 14948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:19.401036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.14.16.67:36659)
0807 19:37:19.401703 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:19.401709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:19.401738 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:37:19.401806 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:19.401813 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:19.402029 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:19.402200 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:19.403589 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.404616 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.406096 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:19.406671 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:19.406835 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:19.411240 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:19.411253 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:19.411268 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:19.411390 (+ 122us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:19.414165 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:19.414789 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:19.414796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:19.414798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:19.414915 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:19.415222 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:19.415225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:19.415227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:19.415750 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:19.415983 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:19.416244 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 19:37:19.416439 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":369,"threads_started":1}
I20260807 19:37:19.418004 14924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:19.521633 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 14792
I20260807 19:37:19.540571 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 14421
I20260807 19:37:19.541788 14485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:19.648871 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 14421
2026-08-07T19:37:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4001 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:37:19 dist-test-slave-07c2 krb5kdc[14953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:37:19 dist-test-slave-07c2 krb5kdc[14953](info): set up 2 sockets
Aug 07 19:37:19 dist-test-slave-07c2 krb5kdc[14953](info): commencing operation
krb5kdc: starting...
W20260807 19:37:20.753958 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.007s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:37:20 dist-test-slave-07c2 krb5kdc[14953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:37:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:37:20.943156 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:41361
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:41361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:21.259232 14968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:21.259536 14968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:21.259699 14968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:21.266685 14968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:21.266817 14968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:21.266935 14968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:21.267031 14968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:21.267097 14968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:21.276664 14968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:41361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:41361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:21.277968 14968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:21.279246 14968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:21.291263 14973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:21.291234 14976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:21.291164 14974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:21.292026 14968 server_base.cc:1034] running on GCE node
I20260807 19:37:21.292827 14968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:21.294178 14968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:21.295390 14968 hybrid_clock.cc:648] HybridClock initialized: now 1786131441295336 us; error 57 us; skew 500 ppm
I20260807 19:37:21.295738 14968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:21 dist-test-slave-07c2 krb5kdc[14953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131441, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:21.302181 14968 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:37:21.303789 14968 webserver.cc:460] Webserver started at http://127.14.16.126:42343/ using document root <none> and password file <none>
I20260807 19:37:21.304464 14968 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:21.304618 14968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:21.304936 14968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:21.307878 14968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "b4452528253f461abb625173b3cd6480"
format_stamp: "Formatted at 2026-08-07 19:37:21 on dist-test-slave-07c2"
server_key: "aee5b0ae2a7ba25ae2457d07cffe7d14"
server_key_iv: "0033917f11188af689da648651749380"
server_key_version: "encryptionkey@0"
I20260807 19:37:21.308652 14968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "b4452528253f461abb625173b3cd6480"
format_stamp: "Formatted at 2026-08-07 19:37:21 on dist-test-slave-07c2"
server_key: "aee5b0ae2a7ba25ae2457d07cffe7d14"
server_key_iv: "0033917f11188af689da648651749380"
server_key_version: "encryptionkey@0"
I20260807 19:37:21.313547 14968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:37:21.316855 14983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:21.318074 14968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:37:21.318254 14968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "b4452528253f461abb625173b3cd6480"
format_stamp: "Formatted at 2026-08-07 19:37:21 on dist-test-slave-07c2"
server_key: "aee5b0ae2a7ba25ae2457d07cffe7d14"
server_key_iv: "0033917f11188af689da648651749380"
server_key_version: "encryptionkey@0"
I20260807 19:37:21.318444 14968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:21.349474 14968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:21.353858 14968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:21.354147 14968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:21.372493 14968 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:41361
I20260807 19:37:21.372529 15035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:41361 every 8 connection(s)
I20260807 19:37:21.373804 14968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:21.377362 15036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:21.379133 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 14968
I20260807 19:37:21.379426 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:21.380247 14401 external_mini_cluster.cc:1428] Setting key 84cf9a8400518870c86f572de5d4573e
I20260807 19:37:21.389986 15036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480: Bootstrap starting.
I20260807 19:37:21.393823 15036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480: Neither blocks nor log segments found. Creating new log.
Aug 07 19:37:21 dist-test-slave-07c2 krb5kdc[14953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:21.394964 15036 log.cc:824] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:21.398196 15036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480: No bootstrap required, opened a new log
I20260807 19:37:21.403434 15039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:21.382268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.14.16.126:41361)
0807 19:37:21.383077 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:21.383089 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:21.383127 (+ 38us) server_negotiation.cc:407] Connection header received
0807 19:37:21.384812 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:21.384832 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:21.385180 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:21.385478 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:21.387369 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:21.388837 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:21.390539 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:21.391170 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:21.391375 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:21.396177 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:21.396188 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:21.396209 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:21.396265 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:21.399400 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:21.400195 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:21.400202 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:21.400208 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:21.400365 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:21.400769 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:21.400772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:21.400774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:21.401269 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:21.401496 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:21.402016 (+ 520us) server_negotiation.cc:299] Negotiation successful
0807 19:37:21.402347 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":378,"threads_started":1}
I20260807 19:37:21.407531 15036 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4452528253f461abb625173b3cd6480" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41361 } }
I20260807 19:37:21.407898 15036 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:21.408011 15036 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4452528253f461abb625173b3cd6480, State: Initialized, Role: FOLLOWER
I20260807 19:37:21.408608 15036 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4452528253f461abb625173b3cd6480" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41361 } }
I20260807 19:37:21.408851 15036 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:21.408996 15036 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:21.409173 15036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:21.411569 15036 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4452528253f461abb625173b3cd6480" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41361 } }
I20260807 19:37:21.412040 15036 leader_election.cc:302] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4452528253f461abb625173b3cd6480; no voters:
I20260807 19:37:21.412623 15036 leader_election.cc:288] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:21.412765 15042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:21.413477 15042 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [term 1 LEADER]: Becoming Leader. State: Replica: b4452528253f461abb625173b3cd6480, State: Running, Role: LEADER
I20260807 19:37:21.414036 15042 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4452528253f461abb625173b3cd6480" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41361 } }
I20260807 19:37:21.414711 15036 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:21.416920 15043 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4452528253f461abb625173b3cd6480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4452528253f461abb625173b3cd6480" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41361 } } }
I20260807 19:37:21.418023 15043 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:21.417459 15044 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4452528253f461abb625173b3cd6480. Latest consensus state: current_term: 1 leader_uuid: "b4452528253f461abb625173b3cd6480" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4452528253f461abb625173b3cd6480" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41361 } } }
I20260807 19:37:21.420148 15044 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:21.420149 15050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:21.425604 15050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:21.434590 15050 catalog_manager.cc:1347] Generated new cluster ID: d565f0f8876843e7a801a0df37fe81a1
I20260807 19:37:21.434721 15050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:21.461575 15050 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:21.462957 15050 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:21.471338 15050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4452528253f461abb625173b3cd6480: Generated new TSK 0
I20260807 19:37:21.472220 15050 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:37:21.527848 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41361
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:21.859135 15063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:21.859460 15063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:21.859656 15063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:21.866700 15063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:21.866799 15063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:21.867010 15063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:37:21.876748 15063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41361
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:21.877983 15063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:21.879334 15063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:21.892313 15069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:21.892515 15071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:21.892553 15073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:21.893399 15063 server_base.cc:1034] running on GCE node
I20260807 19:37:21.893991 15063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:21.894872 15063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:21.896092 15063 hybrid_clock.cc:648] HybridClock initialized: now 1786131441896053 us; error 30 us; skew 500 ppm
I20260807 19:37:21.896402 15063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:21 dist-test-slave-07c2 krb5kdc[14953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131441, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:21.899581 15063 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:37:21.900949 15063 webserver.cc:460] Webserver started at http://127.14.16.65:41385/ using document root <none> and password file <none>
I20260807 19:37:21.901604 15063 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:21.901746 15063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:21.902063 15063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:21.905269 15063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "f219f4c647bf404c90bffe315d175c9d"
format_stamp: "Formatted at 2026-08-07 19:37:21 on dist-test-slave-07c2"
server_key: "3364eaa96f807121143b55ec16536f18"
server_key_iv: "74950120e8f66c785c509a4397232894"
server_key_version: "encryptionkey@0"
I20260807 19:37:21.906042 15063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "f219f4c647bf404c90bffe315d175c9d"
format_stamp: "Formatted at 2026-08-07 19:37:21 on dist-test-slave-07c2"
server_key: "3364eaa96f807121143b55ec16536f18"
server_key_iv: "74950120e8f66c785c509a4397232894"
server_key_version: "encryptionkey@0"
I20260807 19:37:21.910897 15063 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 19:37:21.914166 15080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:21.915551 15063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:37:21.915723 15063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "f219f4c647bf404c90bffe315d175c9d"
format_stamp: "Formatted at 2026-08-07 19:37:21 on dist-test-slave-07c2"
server_key: "3364eaa96f807121143b55ec16536f18"
server_key_iv: "74950120e8f66c785c509a4397232894"
server_key_version: "encryptionkey@0"
I20260807 19:37:21.915916 15063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:21.973001 15063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:21.977550 15063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:21.977916 15063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:21.978899 15063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:21.980293 15063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:21.980393 15063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:21.980535 15063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:21.980607 15063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:22.015252 15063 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:41259
I20260807 19:37:22.015337 15196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:41259 every 8 connection(s)
I20260807 19:37:22.016702 15063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:37:22.018867 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 15063
I20260807 19:37:22.019107 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:37:22.019992 14401 external_mini_cluster.cc:1428] Setting key 194ec08345aa5b0b3e117fc63c794532
Aug 07 19:37:22 dist-test-slave-07c2 krb5kdc[14953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131441, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:22.040280 15199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:22.019397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:41663 (local address 127.14.16.126:41361)
0807 19:37:22.019940 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:22.019947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:22.021174 (+ 1227us) server_negotiation.cc:407] Connection header received
0807 19:37:22.023679 (+ 2505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:22.023683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:22.023748 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:22.023841 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:22.025917 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:22.026645 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:22.029028 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:22.029753 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:22.029907 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:22.033252 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:22.033260 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:22.033265 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:22.033310 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:22.037007 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:22.037698 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:22.037706 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:22.037712 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:22.037894 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:22.038324 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:22.038331 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:22.038336 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:22.038709 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:22.038896 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:22.039593 (+ 697us) server_negotiation.cc:299] Negotiation successful
0807 19:37:22.039848 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":284,"threads_started":1}
I20260807 19:37:22.040864 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:22.020104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41361 (local address 127.14.16.65:41663)
0807 19:37:22.020996 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:22.021058 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:22.023409 (+ 2351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:22.024112 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:22.024126 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:22.024916 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:22.025735 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:22.025763 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:22.026807 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:22.026815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:22.028889 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:22.028901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:22.029952 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:22.029957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:22.030136 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:22.030826 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:22.030848 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:22.033043 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:22.037137 (+ 4094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:22.037144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:22.037155 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:22.037551 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:22.038020 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:22.038023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:22.038026 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:22.038199 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:22.038841 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:22.038846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:22.039140 (+ 294us) client_negotiation.cc:769] Sending connection context
0807 19:37:22.039417 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 19:37:22.039806 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":328,"threads_started":1}
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
I20260807 19:37:22.042557 15197 heartbeater.cc:347] Connected to a master server at 127.14.16.126:41361
I20260807 19:37:22.042933 15197 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
I20260807 19:37:22.043880 15197 heartbeater.cc:511] Master 127.14.16.126:41361 requested a full tablet report, sending...
I20260807 19:37:22.045953 15000 ts_manager.cc:194] Registered new tserver with Master: f219f4c647bf404c90bffe315d175c9d (127.14.16.65:41259)
I20260807 19:37:22.047429 15000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:41663
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:37:22.064574 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41361
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:22.401422 15203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:22.401726 15203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:22.401825 15203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:22.409301 15203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:22.409425 15203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:22.409543 15203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:37:22.419786 15203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41361
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:22.421226 15203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:22.422559 15203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:22.435894 15211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:22.435956 15209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:22.436439 15208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:22.436633 15203 server_base.cc:1034] running on GCE node
I20260807 19:37:22.437242 15203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:22.438207 15203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:22.439443 15203 hybrid_clock.cc:648] HybridClock initialized: now 1786131442439396 us; error 59 us; skew 500 ppm
I20260807 19:37:22.439780 15203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:22 dist-test-slave-07c2 krb5kdc[14953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131442, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:22.443135 15203 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:37:22.444679 15203 webserver.cc:460] Webserver started at http://127.14.16.66:45477/ using document root <none> and password file <none>
I20260807 19:37:22.445394 15203 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:22.445571 15203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:22.445921 15203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:22.448953 15203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "b7fba15ce6ed4a9982d269795bf3ed3e"
format_stamp: "Formatted at 2026-08-07 19:37:22 on dist-test-slave-07c2"
server_key: "39f274938050a01fdccc63101f111ff0"
server_key_iv: "3d8bf7c6e3e43d2a416481f8ecb64618"
server_key_version: "encryptionkey@0"
I20260807 19:37:22.449771 15203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "b7fba15ce6ed4a9982d269795bf3ed3e"
format_stamp: "Formatted at 2026-08-07 19:37:22 on dist-test-slave-07c2"
server_key: "39f274938050a01fdccc63101f111ff0"
server_key_iv: "3d8bf7c6e3e43d2a416481f8ecb64618"
server_key_version: "encryptionkey@0"
I20260807 19:37:22.454957 15203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:37:22.458520 15218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:22.459906 15203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 19:37:22.460089 15203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "b7fba15ce6ed4a9982d269795bf3ed3e"
format_stamp: "Formatted at 2026-08-07 19:37:22 on dist-test-slave-07c2"
server_key: "39f274938050a01fdccc63101f111ff0"
server_key_iv: "3d8bf7c6e3e43d2a416481f8ecb64618"
server_key_version: "encryptionkey@0"
I20260807 19:37:22.460296 15203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:22.476924 15203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:22.481443 15203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:22.481835 15203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:22.482828 15203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:22.484294 15203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:22.484401 15203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:22.484553 15203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:22.484640 15203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:22.520143 15203 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:40677
I20260807 19:37:22.520213 15335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:40677 every 8 connection(s)
I20260807 19:37:22.521755 15203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:37:22.522811 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 15203
I20260807 19:37:22.523025 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:37:22.523665 14401 external_mini_cluster.cc:1428] Setting key 13d85eb9aa7a8a35f6e6493a353b35da
Aug 07 19:37:22 dist-test-slave-07c2 krb5kdc[14953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131442, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
I20260807 19:37:22.544705 15199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:22.524947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:55085 (local address 127.14.16.126:41361)
0807 19:37:22.525143 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:22.525150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:22.526488 (+ 1338us) server_negotiation.cc:407] Connection header received
0807 19:37:22.529471 (+ 2983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:22.529475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:22.529545 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:22.529633 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:22.531789 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:22.532502 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:22.534105 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:22.534556 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:22.534693 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:22.538152 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:22.538157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:22.538160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:22.538193 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:22.541104 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:22.542173 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:22.542178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:22.542180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:22.542313 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:22.542779 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:22.542783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:22.542785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:22.543083 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:22.543270 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:22.544112 (+ 842us) server_negotiation.cc:299] Negotiation successful
0807 19:37:22.544370 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
I20260807 19:37:22.545866 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:22.525466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41361 (local address 127.14.16.66:55085)
0807 19:37:22.526319 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:22.526366 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:22.529173 (+ 2807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:22.529924 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:22.529938 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:22.530753 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:22.531594 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:22.531621 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:22.532659 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:22.532667 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:22.533954 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:22.533966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:22.534792 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:22.534800 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:22.535027 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:22.535780 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:22.535803 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:22.537928 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:22.541424 (+ 3496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:22.541437 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:22.541455 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:22.542015 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:22.542433 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:22.542438 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:22.542442 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:22.542644 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:22.543238 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:22.543246 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:22.543676 (+ 430us) client_negotiation.cc:769] Sending connection context
0807 19:37:22.543950 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 19:37:22.544513 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":340,"threads_started":1}
I20260807 19:37:22.547717 15337 heartbeater.cc:347] Connected to a master server at 127.14.16.126:41361
I20260807 19:37:22.548080 15337 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:22.548842 15337 heartbeater.cc:511] Master 127.14.16.126:41361 requested a full tablet report, sending...
I20260807 19:37:22.550462 15000 ts_manager.cc:194] Registered new tserver with Master: b7fba15ce6ed4a9982d269795bf3ed3e (127.14.16.66:40677)
I20260807 19:37:22.551769 15000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:55085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:37:22.571019 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41361
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:22.897663 15342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:22.897917 15342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:22.898072 15342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:22.905102 15342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:22.905231 15342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:22.905414 15342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:37:22.914716 15342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41361
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:22.915963 15342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:22.917357 15342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:22.930253 15350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:22.930310 15347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:22.930316 15348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:22.931428 15342 server_base.cc:1034] running on GCE node
I20260807 19:37:22.931944 15342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:22.932775 15342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:22.933969 15342 hybrid_clock.cc:648] HybridClock initialized: now 1786131442933912 us; error 49 us; skew 500 ppm
I20260807 19:37:22.934240 15342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:22 dist-test-slave-07c2 krb5kdc[14953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131442, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:22.937451 15342 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:37:22.939000 15342 webserver.cc:460] Webserver started at http://127.14.16.67:37441/ using document root <none> and password file <none>
I20260807 19:37:22.939757 15342 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:22.939913 15342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:22.940232 15342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:22.943064 15342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "520114ad28a34fa9b468ae0b6ab8fcc6"
format_stamp: "Formatted at 2026-08-07 19:37:22 on dist-test-slave-07c2"
server_key: "b6a8173b196c78c2d901386c09e41cb0"
server_key_iv: "547793376c935ded53a6e4588cb0e46a"
server_key_version: "encryptionkey@0"
I20260807 19:37:22.943940 15342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "520114ad28a34fa9b468ae0b6ab8fcc6"
format_stamp: "Formatted at 2026-08-07 19:37:22 on dist-test-slave-07c2"
server_key: "b6a8173b196c78c2d901386c09e41cb0"
server_key_iv: "547793376c935ded53a6e4588cb0e46a"
server_key_version: "encryptionkey@0"
I20260807 19:37:22.949075 15342 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:37:22.952630 15357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:22.953866 15342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:37:22.954185 15342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "520114ad28a34fa9b468ae0b6ab8fcc6"
format_stamp: "Formatted at 2026-08-07 19:37:22 on dist-test-slave-07c2"
server_key: "b6a8173b196c78c2d901386c09e41cb0"
server_key_iv: "547793376c935ded53a6e4588cb0e46a"
server_key_version: "encryptionkey@0"
I20260807 19:37:22.954424 15342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:22.977118 15342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:22.981303 15342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:22.981613 15342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:22.982654 15342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:22.984108 15342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:22.984221 15342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:22.984372 15342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:22.984449 15342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:23.020020 15342 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:37785
I20260807 19:37:23.020198 15473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:37785 every 8 connection(s)
I20260807 19:37:23.021366 15342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:37:23.030287 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 15342
I20260807 19:37:23.030524 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:37:23.030995 14401 external_mini_cluster.cc:1428] Setting key 9c823d11334652e8f32b124623ce369a
Aug 07 19:37:23 dist-test-slave-07c2 krb5kdc[14953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131442, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:23.042004 15199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.024024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:55461 (local address 127.14.16.126:41361)
0807 19:37:23.024217 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.024222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.025431 (+ 1209us) server_negotiation.cc:407] Connection header received
0807 19:37:23.027342 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.027348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.027473 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.027593 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:23.029132 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.029889 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.031696 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.032236 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.032391 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.036162 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:23.036168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:23.036172 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:23.036225 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:23.038797 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.039606 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.039613 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.039617 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.039767 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.040265 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.040274 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.040276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.040547 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:23.040706 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.041408 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.041636 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:37:23.042954 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.024519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41361 (local address 127.14.16.67:55461)
0807 19:37:23.025263 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:23.025304 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:23.027037 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:23.027852 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:23.027860 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:23.028372 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:23.028944 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:23.028961 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.030040 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.030050 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:23.031547 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:23.031559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.032471 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.032481 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:23.032733 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.033445 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:23.033475 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:23.035966 (+ 2491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:23.038942 (+ 2976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:23.038952 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:23.038969 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:23.039487 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:23.039903 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:23.039908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:23.039911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:23.040134 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:23.040679 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:23.040687 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:23.041039 (+ 352us) client_negotiation.cc:769] Sending connection context
0807 19:37:23.041307 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 19:37:23.041864 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":308,"threads_started":1}
I20260807 19:37:23.044404 15474 heartbeater.cc:347] Connected to a master server at 127.14.16.126:41361
I20260807 19:37:23.044739 15474 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:23.045661 15474 heartbeater.cc:511] Master 127.14.16.126:41361 requested a full tablet report, sending...
I20260807 19:37:23.047230 15000 ts_manager.cc:194] Registered new tserver with Master: 520114ad28a34fa9b468ae0b6ab8fcc6 (127.14.16.67:37785)
I20260807 19:37:23.048066 15000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:55461
I20260807 19:37:23.050081 15197 heartbeater.cc:503] Master 127.14.16.126:41361 was elected leader, sending a full tablet report...
I20260807 19:37:23.059020 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:37:23.078176 15199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.065454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51044 (local address 127.14.16.126:41361)
0807 19:37:23.065642 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.065646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.066191 (+ 545us) server_negotiation.cc:407] Connection header received
0807 19:37:23.066331 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.066335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.066391 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.066477 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:23.068089 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.068857 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.070067 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.070468 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.070557 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.072774 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:23.072779 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:23.072782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:23.072811 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:23.075539 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.076125 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.076129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.076131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.076275 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.076768 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.076772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.076774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.077122 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:23.077312 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.077674 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.077840 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:37:23.084965 15000 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:37:23.092192 15000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:37:23.115010 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.104745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.14.16.67:37785)
0807 19:37:23.105434 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.105445 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.106242 (+ 797us) server_negotiation.cc:407] Connection header received
0807 19:37:23.106567 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.106579 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.106878 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.107078 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:23.108153 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.109834 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.112525 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.113834 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.113967 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.114219 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.114491 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.114666 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":330,"threads_started":1}
I20260807 19:37:23.115136 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.104467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:37785 (local address 127.0.0.1:58058)
0807 19:37:23.106081 (+ 1614us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:23.106119 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:23.106329 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:23.107536 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:23.107545 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:23.107582 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:23.107946 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:23.107959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.109977 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.109982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:23.112105 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:23.112117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.114031 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.114038 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:23.114248 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.114300 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 19:37:23.114448 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 19:37:23.114668 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1099,"mutex_wait_us":23,"thread_start_us":960,"threads_started":1}
I20260807 19:37:23.115996 15492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.106248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:40677 (local address 127.0.0.1:41114)
0807 19:37:23.107008 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:23.107025 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:23.107155 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:23.108304 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:23.108310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:23.108339 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:23.108752 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:23.108764 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.110625 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.110631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:23.112872 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:23.112887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.114735 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.114741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:23.114914 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.114952 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:37:23.115034 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 19:37:23.115719 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":72,"thread_start_us":695,"threads_started":1}
I20260807 19:37:23.116533 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.106777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.14.16.66:40677)
0807 19:37:23.107698 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.107708 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.107736 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:37:23.107875 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.107884 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.108099 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.108296 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:23.108957 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.110484 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.113061 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.114499 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.115775 (+ 1276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.115938 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.115999 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.116182 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":651,"threads_started":1}
I20260807 19:37:23.117159 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.104866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:41259 (local address 127.0.0.1:33740)
0807 19:37:23.106085 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:23.106119 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:23.106317 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:23.108956 (+ 2639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:23.108962 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:23.108988 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:23.109409 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:23.109423 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.111280 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.111309 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:23.113470 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:23.113485 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.116325 (+ 2840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.116335 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:23.116517 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.116551 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:37:23.116649 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 19:37:23.116777 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1078,"mutex_wait_us":3,"thread_start_us":930,"threads_started":1}
I20260807 19:37:23.117899 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.105594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33740 (local address 127.14.16.65:41259)
0807 19:37:23.108160 (+ 2566us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.108167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.108197 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:37:23.108291 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.108298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.108514 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.108713 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:23.109596 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.111117 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.113643 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.115202 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.115369 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.115606 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.117393 (+ 1787us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.117574 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2385,"thread_start_us":2165,"threads_started":1}
I20260807 19:37:23.119525 15406 tablet_service.cc:1467] Processing CreateTablet for tablet b74482b4afbe41478d7a912e2a03ab19 (DEFAULT_TABLE table=test-table [id=0bc6b5c569024682a774b7d871bf2707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:23.120051 15270 tablet_service.cc:1467] Processing CreateTablet for tablet b74482b4afbe41478d7a912e2a03ab19 (DEFAULT_TABLE table=test-table [id=0bc6b5c569024682a774b7d871bf2707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:23.120976 15406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b74482b4afbe41478d7a912e2a03ab19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:23.121447 15270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b74482b4afbe41478d7a912e2a03ab19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:23.121317 15131 tablet_service.cc:1467] Processing CreateTablet for tablet b74482b4afbe41478d7a912e2a03ab19 (DEFAULT_TABLE table=test-table [id=0bc6b5c569024682a774b7d871bf2707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:23.122717 15131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b74482b4afbe41478d7a912e2a03ab19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:23.132597 15494 tablet_bootstrap.cc:492] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6: Bootstrap starting.
I20260807 19:37:23.136025 15495 tablet_bootstrap.cc:492] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e: Bootstrap starting.
I20260807 19:37:23.136307 15494 tablet_bootstrap.cc:654] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:23.137725 15494 log.cc:824] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:23.139791 15495 tablet_bootstrap.cc:654] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:23.139854 15497 tablet_bootstrap.cc:492] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d: Bootstrap starting.
I20260807 19:37:23.141079 15494 tablet_bootstrap.cc:492] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6: No bootstrap required, opened a new log
I20260807 19:37:23.141073 15495 log.cc:824] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:23.141369 15494 ts_tablet_manager.cc:1397] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260807 19:37:23.143344 15497 tablet_bootstrap.cc:654] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:23.144358 15495 tablet_bootstrap.cc:492] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e: No bootstrap required, opened a new log
I20260807 19:37:23.144586 15497 log.cc:824] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:23.144644 15495 ts_tablet_manager.cc:1397] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260807 19:37:23.147645 15497 tablet_bootstrap.cc:492] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d: No bootstrap required, opened a new log
I20260807 19:37:23.147914 15497 ts_tablet_manager.cc:1397] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260807 19:37:23.151759 15494 raft_consensus.cc:348] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37785 } } peers { permanent_uuid: "f219f4c647bf404c90bffe315d175c9d" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 41259 } } peers { permanent_uuid: "b7fba15ce6ed4a9982d269795bf3ed3e" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40677 } }
I20260807 19:37:23.152249 15494 raft_consensus.cc:374] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:23.152354 15494 raft_consensus.cc:729] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520114ad28a34fa9b468ae0b6ab8fcc6, State: Initialized, Role: FOLLOWER
I20260807 19:37:23.152215 15495 raft_consensus.cc:348] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37785 } } peers { permanent_uuid: "f219f4c647bf404c90bffe315d175c9d" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 41259 } } peers { permanent_uuid: "b7fba15ce6ed4a9982d269795bf3ed3e" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40677 } }
I20260807 19:37:23.152767 15495 raft_consensus.cc:374] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:23.152882 15495 raft_consensus.cc:729] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fba15ce6ed4a9982d269795bf3ed3e, State: Initialized, Role: FOLLOWER
I20260807 19:37:23.152962 15494 consensus_queue.cc:260] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37785 } } peers { permanent_uuid: "f219f4c647bf404c90bffe315d175c9d" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 41259 } } peers { permanent_uuid: "b7fba15ce6ed4a9982d269795bf3ed3e" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40677 } }
I20260807 19:37:23.153501 15495 consensus_queue.cc:260] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37785 } } peers { permanent_uuid: "f219f4c647bf404c90bffe315d175c9d" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 41259 } } peers { permanent_uuid: "b7fba15ce6ed4a9982d269795bf3ed3e" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40677 } }
I20260807 19:37:23.154320 15474 heartbeater.cc:503] Master 127.14.16.126:41361 was elected leader, sending a full tablet report...
I20260807 19:37:23.154847 15494 ts_tablet_manager.cc:1428] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6: Time spent starting tablet: real 0.013s user 0.006s sys 0.007s
I20260807 19:37:23.155866 15337 heartbeater.cc:503] Master 127.14.16.126:41361 was elected leader, sending a full tablet report...
I20260807 19:37:23.156847 15495 ts_tablet_manager.cc:1428] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260807 19:37:23.159837 15497 raft_consensus.cc:348] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37785 } } peers { permanent_uuid: "f219f4c647bf404c90bffe315d175c9d" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 41259 } } peers { permanent_uuid: "b7fba15ce6ed4a9982d269795bf3ed3e" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40677 } }
I20260807 19:37:23.160338 15497 raft_consensus.cc:374] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:23.160456 15497 raft_consensus.cc:729] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f219f4c647bf404c90bffe315d175c9d, State: Initialized, Role: FOLLOWER
I20260807 19:37:23.161023 15497 consensus_queue.cc:260] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37785 } } peers { permanent_uuid: "f219f4c647bf404c90bffe315d175c9d" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 41259 } } peers { permanent_uuid: "b7fba15ce6ed4a9982d269795bf3ed3e" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40677 } }
I20260807 19:37:23.162752 15497 ts_tablet_manager.cc:1428] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d: Time spent starting tablet: real 0.015s user 0.007s sys 0.006s
W20260807 19:37:23.269179 15198 tablet.cc:2369] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:37:23.273159 15475 tablet.cc:2369] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:37:23.274250 15338 tablet.cc:2369] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:37:23.319132 15500 raft_consensus.cc:482] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:23.319505 15500 raft_consensus.cc:504] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37785 } } peers { permanent_uuid: "f219f4c647bf404c90bffe315d175c9d" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 41259 } } peers { permanent_uuid: "b7fba15ce6ed4a9982d269795bf3ed3e" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40677 } }
I20260807 19:37:23.321533 15500 leader_election.cc:288] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f219f4c647bf404c90bffe315d175c9d (127.14.16.65:41259), b7fba15ce6ed4a9982d269795bf3ed3e (127.14.16.66:40677)
I20260807 19:37:23.327414 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.322119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:46077 (local address 127.14.16.66:40677)
0807 19:37:23.322411 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.322418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.322438 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:37:23.322516 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.322523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.322591 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.322694 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:23.323151 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.323942 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.325733 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.326471 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.326636 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.326712 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.326977 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.327101 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260807 19:37:23.327414 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.321898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:40677 (local address 127.14.16.67:46077)
0807 19:37:23.322196 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:23.322216 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:23.322346 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:23.322718 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:23.322723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:23.322753 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:23.323013 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:23.323023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.324080 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.324085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:23.325580 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:23.325592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.326626 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.326634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:23.326825 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.326848 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:37:23.326951 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 19:37:23.327084 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260807 19:37:23.327944 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.321992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:43115 (local address 127.14.16.65:41259)
0807 19:37:23.322196 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.322204 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.322865 (+ 661us) server_negotiation.cc:407] Connection header received
0807 19:37:23.323138 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.323141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.323199 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.323316 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:23.323869 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.324553 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.326270 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.327017 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.327205 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.327273 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.327623 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.327740 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:37:23.328469 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.321922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:41259 (local address 127.14.16.67:43115)
0807 19:37:23.322747 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:23.322761 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:23.322986 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:23.323340 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:23.323344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:23.323369 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:23.323705 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:23.323713 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.324699 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.324705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:23.326099 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:23.326111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.327221 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.327228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:23.327411 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.327433 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 19:37:23.328121 (+ 688us) client_negotiation.cc:240] Negotiation successful
0807 19:37:23.328248 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":520,"threads_started":1}
I20260807 19:37:23.328385 15290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b74482b4afbe41478d7a912e2a03ab19" candidate_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fba15ce6ed4a9982d269795bf3ed3e" is_pre_election: true
I20260807 19:37:23.328923 15290 raft_consensus.cc:2454] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520114ad28a34fa9b468ae0b6ab8fcc6 in term 0.
I20260807 19:37:23.329519 15151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b74482b4afbe41478d7a912e2a03ab19" candidate_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f219f4c647bf404c90bffe315d175c9d" is_pre_election: true
I20260807 19:37:23.329638 15361 leader_election.cc:302] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520114ad28a34fa9b468ae0b6ab8fcc6, b7fba15ce6ed4a9982d269795bf3ed3e; no voters:
I20260807 19:37:23.330067 15500 raft_consensus.cc:2781] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:37:23.330049 15151 raft_consensus.cc:2454] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520114ad28a34fa9b468ae0b6ab8fcc6 in term 0.
I20260807 19:37:23.330233 15500 raft_consensus.cc:482] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:37:23.330363 15500 raft_consensus.cc:3037] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:23.332780 15500 raft_consensus.cc:504] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37785 } } peers { permanent_uuid: "f219f4c647bf404c90bffe315d175c9d" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 41259 } } peers { permanent_uuid: "b7fba15ce6ed4a9982d269795bf3ed3e" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40677 } }
I20260807 19:37:23.333325 15500 leader_election.cc:288] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [CANDIDATE]: Term 1 election: Requested vote from peers f219f4c647bf404c90bffe315d175c9d (127.14.16.65:41259), b7fba15ce6ed4a9982d269795bf3ed3e (127.14.16.66:40677)
I20260807 19:37:23.333777 15290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b74482b4afbe41478d7a912e2a03ab19" candidate_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7fba15ce6ed4a9982d269795bf3ed3e"
I20260807 19:37:23.333803 15151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b74482b4afbe41478d7a912e2a03ab19" candidate_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f219f4c647bf404c90bffe315d175c9d"
I20260807 19:37:23.333981 15290 raft_consensus.cc:3037] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:23.333988 15151 raft_consensus.cc:3037] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:23.336385 15290 raft_consensus.cc:2454] T b74482b4afbe41478d7a912e2a03ab19 P b7fba15ce6ed4a9982d269795bf3ed3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520114ad28a34fa9b468ae0b6ab8fcc6 in term 1.
I20260807 19:37:23.336391 15151 raft_consensus.cc:2454] T b74482b4afbe41478d7a912e2a03ab19 P f219f4c647bf404c90bffe315d175c9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520114ad28a34fa9b468ae0b6ab8fcc6 in term 1.
I20260807 19:37:23.336815 15361 leader_election.cc:302] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520114ad28a34fa9b468ae0b6ab8fcc6, b7fba15ce6ed4a9982d269795bf3ed3e; no voters:
I20260807 19:37:23.337129 15500 raft_consensus.cc:2781] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:23.337608 15500 raft_consensus.cc:686] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [term 1 LEADER]: Becoming Leader. State: Replica: 520114ad28a34fa9b468ae0b6ab8fcc6, State: Running, Role: LEADER
I20260807 19:37:23.338164 15500 consensus_queue.cc:237] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37785 } } peers { permanent_uuid: "f219f4c647bf404c90bffe315d175c9d" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 41259 } } peers { permanent_uuid: "b7fba15ce6ed4a9982d269795bf3ed3e" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40677 } }
I20260807 19:37:23.342787 15000 catalog_manager.cc:5495] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 520114ad28a34fa9b468ae0b6ab8fcc6 (127.14.16.67). New cstate: current_term: 1 leader_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520114ad28a34fa9b468ae0b6ab8fcc6" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f219f4c647bf404c90bffe315d175c9d" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 41259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7fba15ce6ed4a9982d269795bf3ed3e" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40677 } health_report { overall_health: UNKNOWN } } }
Aug 07 19:37:23 dist-test-slave-07c2 krb5kdc[14953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:37:23.417865 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.402924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33752 (local address 127.14.16.65:41259)
0807 19:37:23.403176 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.403186 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.403690 (+ 504us) server_negotiation.cc:407] Connection header received
0807 19:37:23.403833 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.403837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.403904 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.403987 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:23.405495 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.406239 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.407467 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.407899 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.408004 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.412495 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:23.412513 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:23.412528 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:23.412567 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:23.415451 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.416089 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.416096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.416098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.416244 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.416548 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.416551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.416553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.416904 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:23.417125 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.417336 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.417499 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 19:37:23 dist-test-slave-07c2 krb5kdc[14953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131443, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:37:23 dist-test-slave-07c2 krb5kdc[14953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131443, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:37:23.469415 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.455337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33760 (local address 127.14.16.65:41259)
0807 19:37:23.455586 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.455593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.455928 (+ 335us) server_negotiation.cc:407] Connection header received
0807 19:37:23.456145 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.456154 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.456220 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.456331 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:23.457710 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.458535 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.459959 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.460457 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.460595 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.464660 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:23.464665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:23.464668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:23.464712 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:23.467162 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.467783 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.467787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.467789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.467921 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.468284 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.468287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.468289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.468549 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:23.468761 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.469000 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.469173 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 19:37:23 dist-test-slave-07c2 krb5kdc[14953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:37:23 dist-test-slave-07c2 krb5kdc[14953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:37:23.518472 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.503995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33764 (local address 127.14.16.65:41259)
0807 19:37:23.504275 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.504283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.504588 (+ 305us) server_negotiation.cc:407] Connection header received
0807 19:37:23.504823 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.504827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.504893 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.505005 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:23.506401 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.507098 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.508405 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.508862 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.509000 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.512937 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:23.512942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:23.512946 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:23.512998 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:23.515845 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.516501 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.516508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.516512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.516644 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.517006 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.517012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.517015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.517314 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:23.517588 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.517875 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.518113 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260807 19:37:23.520486 15109 tablet_service.cc:2908] Rejecting scan request for tablet b74482b4afbe41478d7a912e2a03ab19: Uninitialized: clean time has not yet been initialized
I20260807 19:37:23.523727 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 15063
I20260807 19:37:23.537626 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.525002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33774 (local address 127.14.16.65:41259)
0807 19:37:23.525360 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.525366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.525634 (+ 268us) server_negotiation.cc:407] Connection header received
0807 19:37:23.525936 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.525944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.526038 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.526181 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:23.527448 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.528142 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.529365 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.529823 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.529974 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.532355 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:23.532368 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:23.532371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:23.532403 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:23.534992 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.535944 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.535948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.535950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.536083 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.536461 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.536466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.536468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.536771 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:23.537001 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.537236 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.537401 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260807 19:37:23.538376 15192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:23.633082 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 15063
I20260807 19:37:23.653079 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 15203
Aug 07 19:37:23 dist-test-slave-07c2 krb5kdc[14953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:37:23.668674 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.653951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41118 (local address 127.14.16.66:40677)
0807 19:37:23.654148 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.654154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.654176 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:37:23.654340 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.654345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.654664 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.654815 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:23.656189 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.656931 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.658301 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.658703 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.658843 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.662825 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:23.662838 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:23.662853 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:23.662886 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:23.665781 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.666473 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.666481 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.666484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.666632 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.667033 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.667039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.667042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.667469 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:23.667796 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.668081 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.668308 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:37:23.669322 15331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:23.757597 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 15203
W20260807 19:37:23.777232 15361 connection.cc:535] client connection to 127.14.16.66:40677 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 19:37:23.777513 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 15342
W20260807 19:37:23.777583 15361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 19:37:23.779413 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.778913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:40677 (local address 127.14.16.67:57309)
0807 19:37:23.779060 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:23.779196 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.66:40677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:37:23.779662 15361 consensus_peers.cc:487] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 -> Peer b7fba15ce6ed4a9982d269795bf3ed3e (127.14.16.66:40677): Couldn't send request to peer b7fba15ce6ed4a9982d269795bf3ed3e. Status: Network error: Client connection negotiation failed: client connection to 127.14.16.66:40677: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 07 19:37:23 dist-test-slave-07c2 krb5kdc[14953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:37:23.792894 15521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.778444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58064 (local address 127.14.16.67:37785)
0807 19:37:23.778894 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:23.778899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:23.778918 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:37:23.778999 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:23.779004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:23.779080 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:23.779200 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:23.780703 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.781419 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.782669 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:23.783066 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:23.783161 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:23.787086 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:23.787098 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:23.787113 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:23.787150 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:23.790183 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.790813 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.790821 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.790823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.790964 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:23.791349 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:23.791353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:23.791355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:23.791674 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:23.791905 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:23.792233 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 19:37:23.792442 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":226,"threads_started":1}
I20260807 19:37:23.793010 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.792539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:41259 (local address 127.14.16.67:55201)
0807 19:37:23.792764 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:23.792836 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:41259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260807 19:37:23.793594 15469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:23.794066 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:23.793691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:41259 (local address 127.14.16.67:38219)
0807 19:37:23.793851 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:23.793944 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:41259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 19:37:23.881417 15360 consensus_peers.cc:487] T b74482b4afbe41478d7a912e2a03ab19 P 520114ad28a34fa9b468ae0b6ab8fcc6 -> Peer f219f4c647bf404c90bffe315d175c9d (127.14.16.65:41259): Couldn't send request to peer f219f4c647bf404c90bffe315d175c9d. Status: Network error: Client connection negotiation failed: client connection to 127.14.16.65:41259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260807 19:37:23.881632 15360 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.16.65:41259, user_credentials={real_user=slave}} blocked reactor thread for 87373.5us
I20260807 19:37:23.882368 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 15342
I20260807 19:37:23.901518 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 14968
I20260807 19:37:23.902704 15031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:24.011107 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 14968
2026-08-07T19:37:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4361 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:37:24 dist-test-slave-07c2 krb5kdc[15526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:37:24 dist-test-slave-07c2 krb5kdc[15526](info): set up 2 sockets
Aug 07 19:37:24 dist-test-slave-07c2 krb5kdc[15526](info): commencing operation
krb5kdc: starting...
W20260807 19:37:25.118302 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:37:25 dist-test-slave-07c2 krb5kdc[15526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:37:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:37:25.319726 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:34425
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:34425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:25.644060 15539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:25.644411 15539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:25.644590 15539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:25.651847 15539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:25.651996 15539 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 19:37:25.652099 15539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:25.652216 15539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:25.652302 15539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:25.652413 15539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:25.663002 15539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:34425
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:34425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:25.664343 15539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:25.665678 15539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:25.677575 15544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:25.677630 15545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:25.677841 15547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:25.678635 15539 server_base.cc:1034] running on GCE node
I20260807 19:37:25.679455 15539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:25.680812 15539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:25.682024 15539 hybrid_clock.cc:648] HybridClock initialized: now 1786131445681967 us; error 29 us; skew 500 ppm
I20260807 19:37:25.682387 15539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:25 dist-test-slave-07c2 krb5kdc[15526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131445, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:25.685560 15539 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:37:25.687057 15539 webserver.cc:460] Webserver started at http://127.14.16.126:37079/ using document root <none> and password file <none>
I20260807 19:37:25.687803 15539 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:25.687920 15539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:25.688272 15539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:25.691155 15539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "089fd72535ba4d9db54a3155acf7303d"
format_stamp: "Formatted at 2026-08-07 19:37:25 on dist-test-slave-07c2"
server_key: "9e0239a663643182c622ef88c918ca52"
server_key_iv: "80e59834d647c8abe82b57165c7047e8"
server_key_version: "encryptionkey@0"
I20260807 19:37:25.692001 15539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "089fd72535ba4d9db54a3155acf7303d"
format_stamp: "Formatted at 2026-08-07 19:37:25 on dist-test-slave-07c2"
server_key: "9e0239a663643182c622ef88c918ca52"
server_key_iv: "80e59834d647c8abe82b57165c7047e8"
server_key_version: "encryptionkey@0"
I20260807 19:37:25.697002 15539 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:37:25.700382 15556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:25.701637 15539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:37:25.701821 15539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "089fd72535ba4d9db54a3155acf7303d"
format_stamp: "Formatted at 2026-08-07 19:37:25 on dist-test-slave-07c2"
server_key: "9e0239a663643182c622ef88c918ca52"
server_key_iv: "80e59834d647c8abe82b57165c7047e8"
server_key_version: "encryptionkey@0"
I20260807 19:37:25.702039 15539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:25.745292 15539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:25.749222 15539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:25.749511 15539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:25.771118 15539 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:34425
I20260807 19:37:25.771139 15620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:34425 every 8 connection(s)
I20260807 19:37:25.772470 15539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:25.775959 15621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:25.779053 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 15539
I20260807 19:37:25.779338 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:25.779999 14401 external_mini_cluster.cc:1428] Setting key b428138c494e1ba8ec08c5a2e332e078
I20260807 19:37:25.788591 15621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d: Bootstrap starting.
I20260807 19:37:25.792235 15621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:25.793553 15621 log.cc:824] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:37:25 dist-test-slave-07c2 krb5kdc[15526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:25.796787 15621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d: No bootstrap required, opened a new log
I20260807 19:37:25.803591 15624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:25.781877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49724 (local address 127.14.16.126:34425)
0807 19:37:25.782790 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:25.782801 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:25.782841 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:37:25.784539 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:25.784563 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:25.784921 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:25.785243 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:25.786988 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:25.788326 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:25.790053 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:25.790712 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:25.790840 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:25.796007 (+ 5167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:25.796018 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:25.796036 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:25.796084 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:25.799539 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:25.800343 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:25.800349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:25.800354 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:25.800521 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:25.800929 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:25.800932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:25.800934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:25.801453 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:25.801678 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:25.802138 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 19:37:25.802506 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":528,"threads_started":1}
I20260807 19:37:25.805985 15621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089fd72535ba4d9db54a3155acf7303d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 34425 } }
I20260807 19:37:25.806421 15621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:25.806555 15621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089fd72535ba4d9db54a3155acf7303d, State: Initialized, Role: FOLLOWER
I20260807 19:37:25.807106 15621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089fd72535ba4d9db54a3155acf7303d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 34425 } }
I20260807 19:37:25.807391 15621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:25.807554 15621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:25.807734 15621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:25.809932 15621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089fd72535ba4d9db54a3155acf7303d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 34425 } }
I20260807 19:37:25.810408 15621 leader_election.cc:302] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 089fd72535ba4d9db54a3155acf7303d; no voters:
I20260807 19:37:25.810954 15621 leader_election.cc:288] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:25.811136 15626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:25.811863 15626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [term 1 LEADER]: Becoming Leader. State: Replica: 089fd72535ba4d9db54a3155acf7303d, State: Running, Role: LEADER
I20260807 19:37:25.812456 15626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089fd72535ba4d9db54a3155acf7303d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 34425 } }
I20260807 19:37:25.813191 15621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:25.815649 15627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "089fd72535ba4d9db54a3155acf7303d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089fd72535ba4d9db54a3155acf7303d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 34425 } } }
I20260807 19:37:25.815501 15628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 089fd72535ba4d9db54a3155acf7303d. Latest consensus state: current_term: 1 leader_uuid: "089fd72535ba4d9db54a3155acf7303d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089fd72535ba4d9db54a3155acf7303d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 34425 } } }
I20260807 19:37:25.815981 15627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:25.815996 15628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:25.817782 15635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:25.823014 15635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:25.831190 15635 catalog_manager.cc:1347] Generated new cluster ID: 7355d400f6a54be0909417013f88e20b
I20260807 19:37:25.831327 15635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:25.874486 15635 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:25.875887 15635 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:25.888442 15635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d: Generated new TSK 0
I20260807 19:37:25.889261 15635 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:37:25.956919 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:34425
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:26.289852 15650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:26.290109 15650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:26.290294 15650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:26.297410 15650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:26.297536 15650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:26.297690 15650 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:37:26.297843 15650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:37:26.307842 15650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.16.126:34425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:26.309037 15650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:26.310329 15650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:26.323210 15656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:26.323343 15655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:26.323357 15658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:26.324146 15650 server_base.cc:1034] running on GCE node
I20260807 19:37:26.324725 15650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:26.325605 15650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:26.326857 15650 hybrid_clock.cc:648] HybridClock initialized: now 1786131446326786 us; error 43 us; skew 500 ppm
I20260807 19:37:26.327147 15650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:26 dist-test-slave-07c2 krb5kdc[15526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131446, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:26.336975 15650 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:37:26.338454 15650 webserver.cc:460] Webserver started at http://127.14.16.65:38229/ using document root <none> and password file <none>
I20260807 19:37:26.339409 15650 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:26.339548 15650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:26.339886 15650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:26.342759 15650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "c58eafe073884b1993bff2884c85a327"
format_stamp: "Formatted at 2026-08-07 19:37:26 on dist-test-slave-07c2"
server_key: "fd48f0086c05d79f343073f4d2239b54"
server_key_iv: "e430b2fc6a0960338f42a275f18596b3"
server_key_version: "encryptionkey@0"
I20260807 19:37:26.343623 15650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "c58eafe073884b1993bff2884c85a327"
format_stamp: "Formatted at 2026-08-07 19:37:26 on dist-test-slave-07c2"
server_key: "fd48f0086c05d79f343073f4d2239b54"
server_key_iv: "e430b2fc6a0960338f42a275f18596b3"
server_key_version: "encryptionkey@0"
I20260807 19:37:26.348685 15650 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:37:26.352104 15666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:26.353356 15650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:37:26.353538 15650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "c58eafe073884b1993bff2884c85a327"
format_stamp: "Formatted at 2026-08-07 19:37:26 on dist-test-slave-07c2"
server_key: "fd48f0086c05d79f343073f4d2239b54"
server_key_iv: "e430b2fc6a0960338f42a275f18596b3"
server_key_version: "encryptionkey@0"
I20260807 19:37:26.353768 15650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:26.369807 15650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:26.373607 15650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:26.373869 15650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:26.376636 15650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:26.376749 15650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:26.376897 15650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:26.376956 15650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:37:26 dist-test-slave-07c2 krb5kdc[15526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131446, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:26.402235 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:26.376984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:37293 (local address 127.14.16.126:34425)
0807 19:37:26.377479 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:26.377486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:26.378545 (+ 1059us) server_negotiation.cc:407] Connection header received
0807 19:37:26.380541 (+ 1996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:26.380547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:26.380653 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:26.380783 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:26.382959 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.384067 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.386797 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.387544 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.387693 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:26.393539 (+ 5846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:26.393550 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:26.393557 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:26.393609 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:26.397620 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:26.398731 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:26.398740 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:26.398746 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:26.398916 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:26.399781 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:26.399789 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:26.399795 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:26.400205 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:26.400365 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:26.401457 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0807 19:37:26.401742 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":267,"threads_started":1}
I20260807 19:37:26.403878 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:26.377589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:34425 (local address 127.14.16.65:37293)
0807 19:37:26.378384 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:26.378419 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:26.380267 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:26.381006 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:26.381013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:26.381753 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:26.382691 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:26.382726 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.384232 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.384237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:26.386621 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:26.386650 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.387808 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.387815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:26.388171 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:26.389025 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:26.389064 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:26.393215 (+ 4151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:26.397800 (+ 4585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:26.397815 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:26.397833 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:26.398588 (+ 755us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:26.399082 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:26.399089 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:26.399095 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:26.399551 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:26.400371 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:26.400381 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:26.400906 (+ 525us) client_negotiation.cc:769] Sending connection context
0807 19:37:26.401296 (+ 390us) client_negotiation.cc:240] Negotiation successful
0807 19:37:26.401941 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":358,"threads_started":1}
I20260807 19:37:26.418915 15650 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:34167
I20260807 19:37:26.418953 15791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:34167 every 8 connection(s)
I20260807 19:37:26.420536 15650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:37:26.422748 15792 heartbeater.cc:347] Connected to a master server at 127.14.16.126:34425
I20260807 19:37:26.422987 15792 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:26.423684 15792 heartbeater.cc:511] Master 127.14.16.126:34425 requested a full tablet report, sending...
I20260807 19:37:26.425935 15573 ts_manager.cc:194] Registered new tserver with Master: c58eafe073884b1993bff2884c85a327 (127.14.16.65:34167)
I20260807 19:37:26.426388 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:26.411732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:41013 (local address 127.14.16.126:34425)
0807 19:37:26.411960 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:26.411969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:26.412600 (+ 631us) server_negotiation.cc:407] Connection header received
0807 19:37:26.412918 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:26.412922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:26.412996 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:26.413109 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:26.414854 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.415795 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.417598 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.418258 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.418399 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:26.420612 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:26.420620 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:26.420624 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:26.420667 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:26.423757 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:26.424420 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:26.424427 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:26.424430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:26.424562 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:26.425003 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:26.425007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:26.425009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:26.425297 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:26.425429 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:26.425871 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 19:37:26.426064 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:37:26.426524 15775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:26.411741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:34425 (local address 127.14.16.65:41013)
0807 19:37:26.412467 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:26.412494 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:26.412678 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:26.413144 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:26.413149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:26.413888 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:26.414659 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:26.414679 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.416011 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.416018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:26.417449 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:26.417468 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.418461 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.418470 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:26.418665 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:26.419335 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:26.419356 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:26.420404 (+ 1048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:26.423897 (+ 3493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:26.423902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:26.423906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:26.424298 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:26.424674 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:26.424679 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:26.424683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:26.424882 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:26.425424 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:26.425428 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:26.425621 (+ 193us) client_negotiation.cc:769] Sending connection context
0807 19:37:26.425799 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 19:37:26.426121 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":405,"threads_started":1}
I20260807 19:37:26.427831 15573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:37293
I20260807 19:37:26.429070 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 15650
I20260807 19:37:26.429306 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:37:26.430125 14401 external_mini_cluster.cc:1428] Setting key d762da22462ffdb51e1a59def809b17e
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:37:26.470904 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:34425
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:26.798364 15796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:26.798666 15796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:26.798856 15796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:26.806128 15796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:26.806277 15796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:26.806419 15796 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:37:26.806565 15796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:37:26.816533 15796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.16.126:34425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:26.817867 15796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:26.819214 15796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:26.832104 15802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:26.832541 15801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:26.832598 15804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:26.833071 15796 server_base.cc:1034] running on GCE node
I20260807 19:37:26.833638 15796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:26.834513 15796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:26.835803 15796 hybrid_clock.cc:648] HybridClock initialized: now 1786131446835718 us; error 46 us; skew 500 ppm
I20260807 19:37:26.836158 15796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:26 dist-test-slave-07c2 krb5kdc[15526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131446, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:26.839561 15796 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:37:26.841022 15796 webserver.cc:460] Webserver started at http://127.14.16.66:41919/ using document root <none> and password file <none>
I20260807 19:37:26.841754 15796 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:26.841904 15796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:26.842267 15796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:26.845397 15796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "dcfecb9143ff43deaff1fa75fd05df86"
format_stamp: "Formatted at 2026-08-07 19:37:26 on dist-test-slave-07c2"
server_key: "aa93da7a31716396963345aee0ce562b"
server_key_iv: "ce1a915be6d677fed97b3ae743e39e67"
server_key_version: "encryptionkey@0"
I20260807 19:37:26.846264 15796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "dcfecb9143ff43deaff1fa75fd05df86"
format_stamp: "Formatted at 2026-08-07 19:37:26 on dist-test-slave-07c2"
server_key: "aa93da7a31716396963345aee0ce562b"
server_key_iv: "ce1a915be6d677fed97b3ae743e39e67"
server_key_version: "encryptionkey@0"
I20260807 19:37:26.851716 15796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:37:26.854928 15811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:26.856251 15796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:37:26.856416 15796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "dcfecb9143ff43deaff1fa75fd05df86"
format_stamp: "Formatted at 2026-08-07 19:37:26 on dist-test-slave-07c2"
server_key: "aa93da7a31716396963345aee0ce562b"
server_key_iv: "ce1a915be6d677fed97b3ae743e39e67"
server_key_version: "encryptionkey@0"
I20260807 19:37:26.856634 15796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:26.869884 15796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:26.873524 15796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:26.873736 15796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:26.876237 15796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:26.876350 15796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:26.876577 15796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:26.876680 15796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:37:26 dist-test-slave-07c2 krb5kdc[15526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131446, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:26.900669 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:26.876598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:34015 (local address 127.14.16.126:34425)
0807 19:37:26.876801 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:26.876808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:26.878226 (+ 1418us) server_negotiation.cc:407] Connection header received
0807 19:37:26.880280 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:26.880286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:26.880401 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:26.880532 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:26.882974 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.884078 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.887408 (+ 3330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.887962 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.888108 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:26.892840 (+ 4732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:26.892846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:26.892851 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:26.892896 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:26.896331 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:26.897582 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:26.897589 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:26.897595 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:26.897756 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:26.898377 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:26.898385 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:26.898389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:26.898808 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:26.898976 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:26.900106 (+ 1130us) server_negotiation.cc:299] Negotiation successful
0807 19:37:26.900381 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:37:26.902787 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:26.877209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:34425 (local address 127.14.16.66:34015)
0807 19:37:26.878046 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:26.878097 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:26.879969 (+ 1872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:26.880826 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:26.880838 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:26.881761 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:26.882752 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:26.882781 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.884291 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.884299 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:26.887173 (+ 2874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:26.887194 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.888238 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.888249 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:26.888652 (+ 403us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:26.889498 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:26.889523 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:26.892624 (+ 3101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:26.896491 (+ 3867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:26.896502 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:26.896526 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:26.897411 (+ 885us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:26.897874 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:26.897879 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:26.897883 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:26.898208 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:26.898984 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:26.898997 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:26.899612 (+ 615us) client_negotiation.cc:769] Sending connection context
0807 19:37:26.899985 (+ 373us) client_negotiation.cc:240] Negotiation successful
0807 19:37:26.900725 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":347,"threads_started":1}
I20260807 19:37:26.916847 15796 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:36363
I20260807 19:37:26.916934 15935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:36363 every 8 connection(s)
I20260807 19:37:26.918215 15796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:37:26.919967 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 15796
I20260807 19:37:26.920197 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:37:26.920787 14401 external_mini_cluster.cc:1428] Setting key 80b9f0501b5b49bcbc196f84cae47c01
I20260807 19:37:26.921337 15936 heartbeater.cc:347] Connected to a master server at 127.14.16.126:34425
I20260807 19:37:26.921720 15936 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:26.922492 15936 heartbeater.cc:511] Master 127.14.16.126:34425 requested a full tablet report, sending...
I20260807 19:37:26.930222 15573 ts_manager.cc:194] Registered new tserver with Master: dcfecb9143ff43deaff1fa75fd05df86 (127.14.16.66:36363)
I20260807 19:37:26.931420 15573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:34015
I20260807 19:37:26.935850 15938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:26.922158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:34425 (local address 127.14.16.66:36881)
0807 19:37:26.922831 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:26.922855 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:26.923049 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:26.923437 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:26.923442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:26.923889 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:26.924128 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:26.924138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.925652 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.925655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:26.926612 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:26.926623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.927756 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.927762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:26.927902 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:26.928642 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:26.928661 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:26.929408 (+ 747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:26.933180 (+ 3772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:26.933185 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:26.933189 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:26.933588 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:26.933977 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:26.933982 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:26.933985 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:26.934121 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:26.934696 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:26.934701 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:26.934914 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 19:37:26.935077 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 19:37:26.935397 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":602,"threads_started":1}
I20260807 19:37:26.936138 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:26.922570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:36881 (local address 127.14.16.126:34425)
0807 19:37:26.922748 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:26.922754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:26.922940 (+ 186us) server_negotiation.cc:407] Connection header received
0807 19:37:26.923192 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:26.923198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:26.923305 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:26.923412 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:26.924299 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.925508 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.926756 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:26.927512 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:26.927650 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:26.929617 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:26.929625 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:26.929633 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:26.929679 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:26.933036 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:26.933706 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:26.933713 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:26.933717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:26.933855 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:26.934231 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:26.934236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:26.934240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:26.934567 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:26.934718 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:26.935270 (+ 552us) server_negotiation.cc:299] Negotiation successful
0807 19:37:26.935606 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:37:26.967617 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:34425
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:27.294831 15942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:27.295173 15942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:27.295398 15942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:27.302807 15942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:27.302959 15942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:27.303117 15942 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:37:27.303265 15942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:37:27.313522 15942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.16.126:34425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:27.314858 15942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:27.316287 15942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:27.329721 15947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:27.329617 15948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:27.329684 15950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:27.330744 15942 server_base.cc:1034] running on GCE node
I20260807 19:37:27.331367 15942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:27.332216 15942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:27.333468 15942 hybrid_clock.cc:648] HybridClock initialized: now 1786131447333403 us; error 41 us; skew 500 ppm
I20260807 19:37:27.333768 15942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:27 dist-test-slave-07c2 krb5kdc[15526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131447, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:27.337005 15942 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:37:27.338439 15942 webserver.cc:460] Webserver started at http://127.14.16.67:34943/ using document root <none> and password file <none>
I20260807 19:37:27.339112 15942 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:27.339247 15942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:27.339607 15942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:27.342403 15942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "4141b16354d84d0e9063d6c08721c9e1"
format_stamp: "Formatted at 2026-08-07 19:37:27 on dist-test-slave-07c2"
server_key: "5a192d9e6e764d69a03d5504713fc739"
server_key_iv: "f60cb5a0a2519642d171f19d186eed78"
server_key_version: "encryptionkey@0"
I20260807 19:37:27.343191 15942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "4141b16354d84d0e9063d6c08721c9e1"
format_stamp: "Formatted at 2026-08-07 19:37:27 on dist-test-slave-07c2"
server_key: "5a192d9e6e764d69a03d5504713fc739"
server_key_iv: "f60cb5a0a2519642d171f19d186eed78"
server_key_version: "encryptionkey@0"
I20260807 19:37:27.348186 15942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 19:37:27.351447 15957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:27.352730 15942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:37:27.352903 15942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "4141b16354d84d0e9063d6c08721c9e1"
format_stamp: "Formatted at 2026-08-07 19:37:27 on dist-test-slave-07c2"
server_key: "5a192d9e6e764d69a03d5504713fc739"
server_key_iv: "f60cb5a0a2519642d171f19d186eed78"
server_key_version: "encryptionkey@0"
I20260807 19:37:27.353114 15942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:27.369994 15942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:27.374380 15942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:27.374661 15942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:27.377408 15942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:27.377521 15942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:27.377710 15942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:27.377786 15942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:37:27 dist-test-slave-07c2 krb5kdc[15526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131447, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:27.401827 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.377755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:46701 (local address 127.14.16.126:34425)
0807 19:37:27.377966 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:27.377972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:27.379345 (+ 1373us) server_negotiation.cc:407] Connection header received
0807 19:37:27.381338 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:27.381346 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:27.381441 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:27.381549 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:27.383823 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.385202 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.387685 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.388274 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.388409 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.393267 (+ 4858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:27.393277 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:27.393285 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:27.393341 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:27.398037 (+ 4696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:27.398827 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:27.398837 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:27.398839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:27.398974 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:27.399535 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:27.399541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:27.399545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:27.400001 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:27.400174 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:27.400943 (+ 769us) server_negotiation.cc:299] Negotiation successful
0807 19:37:27.401291 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:37:27.402407 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.378257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:34425 (local address 127.14.16.67:46701)
0807 19:37:27.379131 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:27.379174 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:27.381069 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:27.381796 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:27.381803 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:27.382538 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:27.383497 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.383535 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.385416 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.385423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.387536 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.387548 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.388631 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.388643 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.388985 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.389905 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:27.389950 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:27.392991 (+ 3041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:27.398203 (+ 5212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:27.398210 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:27.398222 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:27.398678 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:27.399096 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:27.399101 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:27.399108 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:27.399375 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:27.400156 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:27.400169 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:27.400507 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 19:37:27.400738 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 19:37:27.401109 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":389,"threads_started":1}
I20260807 19:37:27.425359 16019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.409021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:34425 (local address 127.14.16.67:39899)
0807 19:37:27.409618 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:27.409633 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:27.409809 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:27.410135 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:27.410139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:27.410784 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:27.411221 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.411240 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.412966 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.412975 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.415181 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.415201 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.416498 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.416503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.416646 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.417285 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:27.417302 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:27.418104 (+ 802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:27.422590 (+ 4486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:27.422595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:27.422599 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:27.423099 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:27.423625 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:27.423630 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:27.423633 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:27.423787 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:27.424495 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:27.424498 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:27.424662 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 19:37:27.424793 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 19:37:27.425026 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":453,"threads_started":1}
I20260807 19:37:27.425729 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.408826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:39899 (local address 127.14.16.126:34425)
0807 19:37:27.409469 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:27.409478 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:27.409733 (+ 255us) server_negotiation.cc:407] Connection header received
0807 19:37:27.409935 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:27.409947 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:27.410005 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:27.410106 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:27.411482 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.412748 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.415432 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.416317 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.416478 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.418347 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:27.418353 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:27.418358 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:27.418394 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:27.422433 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:27.423223 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:27.423231 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:27.423237 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:27.423460 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:27.423911 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:27.423919 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:27.423924 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:27.424369 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:27.424577 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:27.424951 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 19:37:27.425252 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458}
I20260807 19:37:27.431253 15792 heartbeater.cc:503] Master 127.14.16.126:34425 was elected leader, sending a full tablet report...
I20260807 19:37:27.438930 15942 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:35955
I20260807 19:37:27.438967 16081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:35955 every 8 connection(s)
I20260807 19:37:27.440135 15942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:37:27.442101 16082 heartbeater.cc:347] Connected to a master server at 127.14.16.126:34425
I20260807 19:37:27.442344 16082 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:27.442981 16082 heartbeater.cc:511] Master 127.14.16.126:34425 requested a full tablet report, sending...
I20260807 19:37:27.444274 15573 ts_manager.cc:194] Registered new tserver with Master: 4141b16354d84d0e9063d6c08721c9e1 (127.14.16.67:35955)
I20260807 19:37:27.445062 15573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:46701
I20260807 19:37:27.446183 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 15942
I20260807 19:37:27.446370 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:37:27.446897 14401 external_mini_cluster.cc:1428] Setting key 703307b4445c67438a177f2e5b15ed13
I20260807 19:37:27.449682 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:37:27 dist-test-slave-07c2 krb5kdc[15526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:37:27 dist-test-slave-07c2 krb5kdc[15526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:27.503080 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.488898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49732 (local address 127.14.16.126:34425)
0807 19:37:27.489188 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:27.489195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:27.489438 (+ 243us) server_negotiation.cc:407] Connection header received
0807 19:37:27.489677 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:27.489682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:27.489750 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:27.489860 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:27.491340 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.492079 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.493355 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.493788 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.493935 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.497958 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:27.497963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:27.497966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:27.498001 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:27.500708 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:27.501324 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:27.501327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:27.501329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:27.501454 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:27.501886 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:27.501890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:27.501892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:27.502179 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:27.502375 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:27.502635 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 19:37:27.502814 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 19:37:27.507582 15573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:37:27.514432 15573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:37:27.545542 16097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.526915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.14.16.66:36363)
0807 19:37:27.528116 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:27.528125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:27.529125 (+ 1000us) server_negotiation.cc:407] Connection header received
0807 19:37:27.536637 (+ 7512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:27.536650 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:27.536991 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:27.537194 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:27.538413 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.540182 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.542801 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.544323 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.544462 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.544665 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:27.545023 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 19:37:27.545208 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1046,"thread_start_us":917,"threads_started":1}
I20260807 19:37:27.545557 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.526668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:36363 (local address 127.0.0.1:38004)
0807 19:37:27.528935 (+ 2267us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:27.528986 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:27.536302 (+ 7316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:27.537603 (+ 1301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:27.537614 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:27.537657 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:27.538158 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.538174 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.540326 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.540331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.542622 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.542637 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.544550 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.544556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.544759 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.544809 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 19:37:27.544954 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 19:37:27.545169 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1931,"thread_start_us":1716,"threads_started":1}
I20260807 19:37:27.547982 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.527606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:35955 (local address 127.0.0.1:54158)
0807 19:37:27.536223 (+ 8617us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:27.536242 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:27.536350 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:27.540783 (+ 4433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:27.540790 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:27.540820 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:27.541337 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.541347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.543041 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.543047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.545200 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.545214 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.547145 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.547155 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.547353 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.547386 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:37:27.547470 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 19:37:27.547606 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8473,"thread_start_us":8332,"threads_started":1}
I20260807 19:37:27.548187 16101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.529640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51742 (local address 127.14.16.65:34167)
0807 19:37:27.539032 (+ 9392us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:27.539041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:27.540081 (+ 1040us) server_negotiation.cc:407] Connection header received
0807 19:37:27.540183 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:27.540188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:27.540428 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:27.540645 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:27.541473 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.542953 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.545742 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.546900 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.547048 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.547253 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:27.547732 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 19:37:27.547890 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9209,"thread_start_us":9350,"threads_started":1}
I20260807 19:37:27.548651 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.529311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:34167 (local address 127.0.0.1:51742)
0807 19:37:27.539802 (+ 10491us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:27.539828 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:27.539957 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:27.540654 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:27.540667 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:27.540698 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:27.541277 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.541294 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.543283 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.543289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.545534 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.545553 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.547423 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.547429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.547550 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.547573 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:37:27.547679 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 19:37:27.547828 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10327,"thread_start_us":10157,"threads_started":1}
I20260807 19:37:27.549460 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.528263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54158 (local address 127.14.16.67:35955)
0807 19:37:27.540235 (+ 11972us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:27.540244 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:27.540279 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:37:27.540381 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:27.540386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:27.540540 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:27.541533 (+ 993us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:27.541791 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.542874 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.545395 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.546837 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.548040 (+ 1203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.548272 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:27.548349 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 19:37:27.549143 (+ 794us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11786,"thread_start_us":11592,"threads_started":1}
I20260807 19:37:27.549995 15865 tablet_service.cc:1467] Processing CreateTablet for tablet fc0874c7612646dd8e25b6569d4b22cd (DEFAULT_TABLE table=test-table [id=8bf157e2467c46329271678310ecbf2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:27.551636 15865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc0874c7612646dd8e25b6569d4b22cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:27.552291 16013 tablet_service.cc:1467] Processing CreateTablet for tablet fc0874c7612646dd8e25b6569d4b22cd (DEFAULT_TABLE table=test-table [id=8bf157e2467c46329271678310ecbf2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:27.552336 15718 tablet_service.cc:1467] Processing CreateTablet for tablet fc0874c7612646dd8e25b6569d4b22cd (DEFAULT_TABLE table=test-table [id=8bf157e2467c46329271678310ecbf2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:27.553400 16013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc0874c7612646dd8e25b6569d4b22cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:27.553617 15718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc0874c7612646dd8e25b6569d4b22cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:27.565243 16102 tablet_bootstrap.cc:492] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1: Bootstrap starting.
I20260807 19:37:27.567198 16103 tablet_bootstrap.cc:492] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86: Bootstrap starting.
I20260807 19:37:27.568833 16104 tablet_bootstrap.cc:492] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327: Bootstrap starting.
I20260807 19:37:27.569479 16102 tablet_bootstrap.cc:654] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:27.570696 16103 tablet_bootstrap.cc:654] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:27.570818 16102 log.cc:824] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:27.571857 16104 tablet_bootstrap.cc:654] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:27.571926 16103 log.cc:824] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:27.573134 16104 log.cc:824] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:27.580262 16102 tablet_bootstrap.cc:492] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1: No bootstrap required, opened a new log
I20260807 19:37:27.580622 16102 ts_tablet_manager.cc:1397] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.006s
I20260807 19:37:27.580706 16103 tablet_bootstrap.cc:492] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86: No bootstrap required, opened a new log
I20260807 19:37:27.581012 16103 ts_tablet_manager.cc:1397] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.007s
I20260807 19:37:27.582149 16104 tablet_bootstrap.cc:492] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327: No bootstrap required, opened a new log
I20260807 19:37:27.582477 16104 ts_tablet_manager.cc:1397] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260807 19:37:27.589619 16103 raft_consensus.cc:348] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.590013 16103 raft_consensus.cc:374] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:27.590168 16103 raft_consensus.cc:729] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcfecb9143ff43deaff1fa75fd05df86, State: Initialized, Role: FOLLOWER
I20260807 19:37:27.590801 16103 consensus_queue.cc:260] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.591939 15936 heartbeater.cc:503] Master 127.14.16.126:34425 was elected leader, sending a full tablet report...
I20260807 19:37:27.591835 16102 raft_consensus.cc:348] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.592244 16102 raft_consensus.cc:374] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:27.592330 16102 raft_consensus.cc:729] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4141b16354d84d0e9063d6c08721c9e1, State: Initialized, Role: FOLLOWER
I20260807 19:37:27.592607 16103 ts_tablet_manager.cc:1428] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86: Time spent starting tablet: real 0.011s user 0.005s sys 0.005s
I20260807 19:37:27.593003 16102 consensus_queue.cc:260] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.593185 16104 raft_consensus.cc:348] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.593729 16104 raft_consensus.cc:374] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:27.593878 16104 raft_consensus.cc:729] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c58eafe073884b1993bff2884c85a327, State: Initialized, Role: FOLLOWER
I20260807 19:37:27.594568 16104 consensus_queue.cc:260] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.596639 16104 ts_tablet_manager.cc:1428] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 19:37:27.598086 16082 heartbeater.cc:503] Master 127.14.16.126:34425 was elected leader, sending a full tablet report...
I20260807 19:37:27.598671 16102 ts_tablet_manager.cc:1428] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1: Time spent starting tablet: real 0.018s user 0.008s sys 0.009s
W20260807 19:37:27.670926 15937 tablet.cc:2369] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:37:27.673715 15793 tablet.cc:2369] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:37:27.691793 16083 tablet.cc:2369] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:37:27.772624 16108 raft_consensus.cc:482] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:27.772979 16108 raft_consensus.cc:504] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.774748 16108 leader_election.cc:288] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c58eafe073884b1993bff2884c85a327 (127.14.16.65:34167), 4141b16354d84d0e9063d6c08721c9e1 (127.14.16.67:35955)
I20260807 19:37:27.783067 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.775426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:38075 (local address 127.14.16.67:35955)
0807 19:37:27.775634 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:27.775639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:27.776610 (+ 971us) server_negotiation.cc:407] Connection header received
0807 19:37:27.776875 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:27.776880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:27.776954 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:27.777040 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:27.777749 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.778826 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.781245 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.782007 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.782207 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.782290 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:27.782678 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 19:37:27.782818 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:37:27.783288 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.775173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:35955 (local address 127.14.16.66:38075)
0807 19:37:27.776511 (+ 1338us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:27.776528 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:27.776716 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:27.777090 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:27.777095 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:27.777121 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:27.777579 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.777591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.778973 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.778980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.781104 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.781114 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.782250 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.782256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.782470 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.782503 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:37:27.782626 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 19:37:27.782767 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1205,"mutex_wait_us":87,"thread_start_us":982,"threads_started":1}
I20260807 19:37:27.783385 16101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.775421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:46169 (local address 127.14.16.65:34167)
0807 19:37:27.775627 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:27.775634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:27.776392 (+ 758us) server_negotiation.cc:407] Connection header received
0807 19:37:27.776690 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:27.776694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:27.776760 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:27.776877 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:27.777597 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.778362 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.780927 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.782155 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.782324 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.782392 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:27.782810 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 19:37:27.782957 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 19:37:27.784039 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.774994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:34167 (local address 127.14.16.66:46169)
0807 19:37:27.776273 (+ 1279us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:27.776293 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:27.776509 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:27.776913 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:27.776917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:27.776942 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:27.777414 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.777428 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.778495 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.778503 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.780737 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:27.780752 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.782452 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.782458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:27.782640 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.782668 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 19:37:27.783564 (+ 896us) client_negotiation.cc:240] Negotiation successful
0807 19:37:27.783719 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1118,"mutex_wait_us":4,"thread_start_us":833,"threads_started":1}
I20260807 19:37:27.784536 16037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0874c7612646dd8e25b6569d4b22cd" candidate_uuid: "dcfecb9143ff43deaff1fa75fd05df86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4141b16354d84d0e9063d6c08721c9e1" is_pre_election: true
I20260807 19:37:27.785097 16037 raft_consensus.cc:2454] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcfecb9143ff43deaff1fa75fd05df86 in term 0.
I20260807 19:37:27.785055 15739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0874c7612646dd8e25b6569d4b22cd" candidate_uuid: "dcfecb9143ff43deaff1fa75fd05df86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c58eafe073884b1993bff2884c85a327" is_pre_election: true
I20260807 19:37:27.785537 15739 raft_consensus.cc:2454] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcfecb9143ff43deaff1fa75fd05df86 in term 0.
I20260807 19:37:27.785902 15813 leader_election.cc:302] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4141b16354d84d0e9063d6c08721c9e1, dcfecb9143ff43deaff1fa75fd05df86; no voters:
I20260807 19:37:27.786371 16108 raft_consensus.cc:2781] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:37:27.786530 16108 raft_consensus.cc:482] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:37:27.786628 16108 raft_consensus.cc:3037] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:27.789700 16108 raft_consensus.cc:504] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.790244 16108 leader_election.cc:288] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [CANDIDATE]: Term 1 election: Requested vote from peers c58eafe073884b1993bff2884c85a327 (127.14.16.65:34167), 4141b16354d84d0e9063d6c08721c9e1 (127.14.16.67:35955)
I20260807 19:37:27.790784 16037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0874c7612646dd8e25b6569d4b22cd" candidate_uuid: "dcfecb9143ff43deaff1fa75fd05df86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4141b16354d84d0e9063d6c08721c9e1"
I20260807 19:37:27.790777 15739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0874c7612646dd8e25b6569d4b22cd" candidate_uuid: "dcfecb9143ff43deaff1fa75fd05df86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c58eafe073884b1993bff2884c85a327"
I20260807 19:37:27.790993 16037 raft_consensus.cc:3037] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:27.790993 15739 raft_consensus.cc:3037] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:27.794240 15739 raft_consensus.cc:2454] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcfecb9143ff43deaff1fa75fd05df86 in term 1.
I20260807 19:37:27.794250 16037 raft_consensus.cc:2454] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcfecb9143ff43deaff1fa75fd05df86 in term 1.
I20260807 19:37:27.794682 15813 leader_election.cc:302] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4141b16354d84d0e9063d6c08721c9e1, c58eafe073884b1993bff2884c85a327, dcfecb9143ff43deaff1fa75fd05df86; no voters:
I20260807 19:37:27.795033 16108 raft_consensus.cc:2781] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:27.795579 16108 raft_consensus.cc:686] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 1 LEADER]: Becoming Leader. State: Replica: dcfecb9143ff43deaff1fa75fd05df86, State: Running, Role: LEADER
I20260807 19:37:27.796123 16108 consensus_queue.cc:237] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.800873 15572 catalog_manager.cc:5495] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 reported cstate change: term changed from 0 to 1, leader changed from <none> to dcfecb9143ff43deaff1fa75fd05df86 (127.14.16.66). New cstate: current_term: 1 leader_uuid: "dcfecb9143ff43deaff1fa75fd05df86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } health_report { overall_health: UNKNOWN } } }
I20260807 19:37:27.844274 16097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:27.838570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.14.16.66:36363)
0807 19:37:27.838799 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:27.838805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:27.838927 (+ 122us) server_negotiation.cc:407] Connection header received
0807 19:37:27.839278 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:27.839320 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:27.839442 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:27.839603 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:27.839966 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.840695 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.842043 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:27.842486 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:27.842615 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:27.843119 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:27.843642 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:27.843798 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:27.843925 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 19:37:27.844008 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:37:27.851720 15739 raft_consensus.cc:1261] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327 [term 1 FOLLOWER]: Refusing update from remote peer dcfecb9143ff43deaff1fa75fd05df86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:27.851729 16037 raft_consensus.cc:1261] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1 [term 1 FOLLOWER]: Refusing update from remote peer dcfecb9143ff43deaff1fa75fd05df86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:27.852641 16113 consensus_queue.cc:1035] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:27.852988 16108 consensus_queue.cc:1035] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:27.867744 16117 mvcc.cc:205] Tried to move back new op lower bound from 7315994410391474176 to 7315994410177220608. Current Snapshot: MvccSnapshot[applied={T|T < 7315994410391474176}]
I20260807 19:37:27.885048 15572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:27.885385 15572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:27.889012 15572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d: Sending DeleteTablet for 3 replicas of tablet fc0874c7612646dd8e25b6569d4b22cd
I20260807 19:37:27.890224 16013 tablet_service.cc:1514] Processing DeleteTablet for tablet fc0874c7612646dd8e25b6569d4b22cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:27 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:54158
I20260807 19:37:27.890220 15718 tablet_service.cc:1514] Processing DeleteTablet for tablet fc0874c7612646dd8e25b6569d4b22cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:27 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:51742
I20260807 19:37:27.891543 15865 tablet_service.cc:1514] Processing DeleteTablet for tablet fc0874c7612646dd8e25b6569d4b22cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:27 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:38004
I20260807 19:37:27.892395 15573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:37:27.893234 15573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:37:27.895814 16129 tablet_replica.cc:330] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86: stopping tablet replica
I20260807 19:37:27.899788 16129 raft_consensus.cc:2229] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:37:27.901625 16012 tablet_service.cc:1467] Processing CreateTablet for tablet 9aff0bb9ef494a4a84ca62d5ef4b9318 (DEFAULT_TABLE table=test-table [id=febc68ab29bd4e4ea7e5172cd3c7b773]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:27.902179 16012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aff0bb9ef494a4a84ca62d5ef4b9318. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:27.902149 16129 raft_consensus.cc:2258] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:27.903677 15717 tablet_service.cc:1467] Processing CreateTablet for tablet 9aff0bb9ef494a4a84ca62d5ef4b9318 (DEFAULT_TABLE table=test-table [id=febc68ab29bd4e4ea7e5172cd3c7b773]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:27.903825 15865 tablet_service.cc:1467] Processing CreateTablet for tablet 9aff0bb9ef494a4a84ca62d5ef4b9318 (DEFAULT_TABLE table=test-table [id=febc68ab29bd4e4ea7e5172cd3c7b773]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:27.904269 15865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aff0bb9ef494a4a84ca62d5ef4b9318. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:27.904457 15717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aff0bb9ef494a4a84ca62d5ef4b9318. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:27.915793 16103 tablet_bootstrap.cc:492] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86: Bootstrap starting.
I20260807 19:37:27.917582 16103 tablet_bootstrap.cc:654] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:27.920789 16104 tablet_bootstrap.cc:492] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327: Bootstrap starting.
I20260807 19:37:27.923038 16104 tablet_bootstrap.cc:654] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:27.928472 16129 ts_tablet_manager.cc:1905] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:27.929793 16102 tablet_bootstrap.cc:492] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1: Bootstrap starting.
I20260807 19:37:27.931313 16103 tablet_bootstrap.cc:492] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86: No bootstrap required, opened a new log
I20260807 19:37:27.931463 16103 ts_tablet_manager.cc:1397] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.002s
I20260807 19:37:27.931658 16102 tablet_bootstrap.cc:654] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:27.932161 16127 tablet_replica.cc:330] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1: stopping tablet replica
I20260807 19:37:27.932585 16127 raft_consensus.cc:2229] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:27.933024 16127 raft_consensus.cc:2258] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:27.934948 16127 ts_tablet_manager.cc:1905] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:27.936633 16103 raft_consensus.cc:348] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.936950 16103 raft_consensus.cc:374] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:27.937070 16103 raft_consensus.cc:729] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcfecb9143ff43deaff1fa75fd05df86, State: Initialized, Role: FOLLOWER
I20260807 19:37:27.937384 16103 consensus_queue.cc:260] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.937520 16128 tablet_replica.cc:330] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327: stopping tablet replica
I20260807 19:37:27.938009 16128 raft_consensus.cc:2229] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:27.938059 16103 ts_tablet_manager.cc:1428] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260807 19:37:27.939539 16128 raft_consensus.cc:2258] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:27.939678 16102 tablet_bootstrap.cc:492] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1: No bootstrap required, opened a new log
I20260807 19:37:27.939826 16102 ts_tablet_manager.cc:1397] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260807 19:37:27.938680 16104 tablet_bootstrap.cc:492] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327: No bootstrap required, opened a new log
I20260807 19:37:27.940676 16104 ts_tablet_manager.cc:1397] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.006s
I20260807 19:37:27.940585 16102 raft_consensus.cc:348] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.940819 16102 raft_consensus.cc:374] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:27.940923 16102 raft_consensus.cc:729] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4141b16354d84d0e9063d6c08721c9e1, State: Initialized, Role: FOLLOWER
I20260807 19:37:27.941366 16102 consensus_queue.cc:260] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.942178 16102 ts_tablet_manager.cc:1428] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:37:27.942710 16104 raft_consensus.cc:348] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.943094 16104 raft_consensus.cc:374] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:27.943220 16104 raft_consensus.cc:729] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c58eafe073884b1993bff2884c85a327, State: Initialized, Role: FOLLOWER
I20260807 19:37:27.943563 16104 consensus_queue.cc:260] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.946094 16104 ts_tablet_manager.cc:1428] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20260807 19:37:27.952150 16127 ts_tablet_manager.cc:1918] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:37:27.952353 16127 log.cc:1196] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/fc0874c7612646dd8e25b6569d4b22cd
I20260807 19:37:27.952929 16127 ts_tablet_manager.cc:1939] T fc0874c7612646dd8e25b6569d4b22cd P 4141b16354d84d0e9063d6c08721c9e1: Deleting consensus metadata
I20260807 19:37:27.954368 16128 ts_tablet_manager.cc:1905] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:27.954706 15558 catalog_manager.cc:4841] TS 4141b16354d84d0e9063d6c08721c9e1 (127.14.16.67:35955): tablet fc0874c7612646dd8e25b6569d4b22cd (table test-table [id=8bf157e2467c46329271678310ecbf2b]) successfully deleted
I20260807 19:37:27.955724 16129 ts_tablet_manager.cc:1918] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:37:27.955972 16129 log.cc:1196] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/fc0874c7612646dd8e25b6569d4b22cd
I20260807 19:37:27.956681 16129 ts_tablet_manager.cc:1939] T fc0874c7612646dd8e25b6569d4b22cd P dcfecb9143ff43deaff1fa75fd05df86: Deleting consensus metadata
I20260807 19:37:27.958421 15560 catalog_manager.cc:4841] TS dcfecb9143ff43deaff1fa75fd05df86 (127.14.16.66:36363): tablet fc0874c7612646dd8e25b6569d4b22cd (table test-table [id=8bf157e2467c46329271678310ecbf2b]) successfully deleted
I20260807 19:37:27.961122 16128 ts_tablet_manager.cc:1918] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:37:27.961282 16128 log.cc:1196] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/fc0874c7612646dd8e25b6569d4b22cd
I20260807 19:37:27.961786 16128 ts_tablet_manager.cc:1939] T fc0874c7612646dd8e25b6569d4b22cd P c58eafe073884b1993bff2884c85a327: Deleting consensus metadata
I20260807 19:37:27.963191 15560 catalog_manager.cc:4841] TS c58eafe073884b1993bff2884c85a327 (127.14.16.65:34167): tablet fc0874c7612646dd8e25b6569d4b22cd (table test-table [id=8bf157e2467c46329271678310ecbf2b]) successfully deleted
I20260807 19:37:27.991029 16122 raft_consensus.cc:482] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:27.991304 16122 raft_consensus.cc:504] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.992250 16122 leader_election.cc:288] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c58eafe073884b1993bff2884c85a327 (127.14.16.65:34167), 4141b16354d84d0e9063d6c08721c9e1 (127.14.16.67:35955)
I20260807 19:37:27.992849 15739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aff0bb9ef494a4a84ca62d5ef4b9318" candidate_uuid: "dcfecb9143ff43deaff1fa75fd05df86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c58eafe073884b1993bff2884c85a327" is_pre_election: true
I20260807 19:37:27.992874 16037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aff0bb9ef494a4a84ca62d5ef4b9318" candidate_uuid: "dcfecb9143ff43deaff1fa75fd05df86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4141b16354d84d0e9063d6c08721c9e1" is_pre_election: true
I20260807 19:37:27.993110 15739 raft_consensus.cc:2454] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcfecb9143ff43deaff1fa75fd05df86 in term 0.
I20260807 19:37:27.993124 16037 raft_consensus.cc:2454] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcfecb9143ff43deaff1fa75fd05df86 in term 0.
I20260807 19:37:27.993569 15816 leader_election.cc:302] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c58eafe073884b1993bff2884c85a327, dcfecb9143ff43deaff1fa75fd05df86; no voters:
I20260807 19:37:27.993918 16122 raft_consensus.cc:2781] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:37:27.994050 16122 raft_consensus.cc:482] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:37:27.994174 16122 raft_consensus.cc:3037] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:27.997442 16122 raft_consensus.cc:504] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:27.998082 16122 leader_election.cc:288] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [CANDIDATE]: Term 1 election: Requested vote from peers c58eafe073884b1993bff2884c85a327 (127.14.16.65:34167), 4141b16354d84d0e9063d6c08721c9e1 (127.14.16.67:35955)
I20260807 19:37:27.998545 16037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aff0bb9ef494a4a84ca62d5ef4b9318" candidate_uuid: "dcfecb9143ff43deaff1fa75fd05df86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4141b16354d84d0e9063d6c08721c9e1"
I20260807 19:37:27.998553 15739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aff0bb9ef494a4a84ca62d5ef4b9318" candidate_uuid: "dcfecb9143ff43deaff1fa75fd05df86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c58eafe073884b1993bff2884c85a327"
I20260807 19:37:27.998752 15739 raft_consensus.cc:3037] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:27.998773 16037 raft_consensus.cc:3037] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:28.000818 16037 raft_consensus.cc:2454] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcfecb9143ff43deaff1fa75fd05df86 in term 1.
I20260807 19:37:28.001273 15813 leader_election.cc:302] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4141b16354d84d0e9063d6c08721c9e1, dcfecb9143ff43deaff1fa75fd05df86; no voters:
I20260807 19:37:28.001576 16122 raft_consensus.cc:2781] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:28.001741 16122 raft_consensus.cc:686] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 1 LEADER]: Becoming Leader. State: Replica: dcfecb9143ff43deaff1fa75fd05df86, State: Running, Role: LEADER
I20260807 19:37:28.001746 15739 raft_consensus.cc:2454] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcfecb9143ff43deaff1fa75fd05df86 in term 1.
I20260807 19:37:28.002066 16122 consensus_queue.cc:237] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:28.004886 15572 catalog_manager.cc:5495] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 reported cstate change: term changed from 0 to 1, leader changed from <none> to dcfecb9143ff43deaff1fa75fd05df86 (127.14.16.66). New cstate: current_term: 1 leader_uuid: "dcfecb9143ff43deaff1fa75fd05df86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } health_report { overall_health: UNKNOWN } } }
Aug 07 19:37:28 dist-test-slave-07c2 krb5kdc[15526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131445, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:28.048924 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.033905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:34425 (local address 127.0.0.1:49740)
0807 19:37:28.034743 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:28.034759 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:28.034933 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:28.035277 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:28.035281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:28.036100 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:28.036400 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.036411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.037555 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.037564 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.038620 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.038633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.039485 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.039492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.039695 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.040443 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:28.040462 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:28.042889 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:28.046179 (+ 3290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:28.046194 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:28.046211 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:28.046622 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:28.047010 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:28.047017 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:28.047021 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:28.047174 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:28.047773 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:28.047782 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:28.047971 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 19:37:28.048140 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 19:37:28.048447 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":544,"threads_started":1}
I20260807 19:37:28.049072 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.033805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49740 (local address 127.14.16.126:34425)
0807 19:37:28.034555 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.034563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.034850 (+ 287us) server_negotiation.cc:407] Connection header received
0807 19:37:28.035073 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.035076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.035131 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.035255 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:28.036564 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.037417 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.038777 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.039226 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.039419 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.043106 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:28.043111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:28.043115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:28.043145 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:28.046043 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:28.046752 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:28.046759 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:28.046763 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:28.046894 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:28.047332 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:28.047343 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:28.047346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:28.047614 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:28.047799 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.048230 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.048621 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":149,"server-negotiator.queue_time_us":542,"thread_start_us":402,"threads_started":1}
I20260807 19:37:28.054368 15572 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:49740:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 19:37:28.055498 15572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:37:28.063040 15718 tablet_service.cc:1467] Processing CreateTablet for tablet b64e8d2af93842f2b4edb00df548ff31 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e7d2698168d24c29bf225be49a9075cf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:37:28.063397 15865 tablet_service.cc:1467] Processing CreateTablet for tablet b64e8d2af93842f2b4edb00df548ff31 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e7d2698168d24c29bf225be49a9075cf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:37:28.063697 15718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b64e8d2af93842f2b4edb00df548ff31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:28.063956 15865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b64e8d2af93842f2b4edb00df548ff31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:28.069402 16013 tablet_service.cc:1467] Processing CreateTablet for tablet b64e8d2af93842f2b4edb00df548ff31 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e7d2698168d24c29bf225be49a9075cf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:37:28.070119 16013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b64e8d2af93842f2b4edb00df548ff31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:28.070709 16103 tablet_bootstrap.cc:492] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86: Bootstrap starting.
I20260807 19:37:28.072796 16103 tablet_bootstrap.cc:654] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:28.074867 16104 tablet_bootstrap.cc:492] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: Bootstrap starting.
I20260807 19:37:28.076702 16104 tablet_bootstrap.cc:654] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:28.077482 16102 tablet_bootstrap.cc:492] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1: Bootstrap starting.
I20260807 19:37:28.078765 16103 tablet_bootstrap.cc:492] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86: No bootstrap required, opened a new log
I20260807 19:37:28.079034 16103 ts_tablet_manager.cc:1397] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 19:37:28.079666 16102 tablet_bootstrap.cc:654] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:28.080144 16103 raft_consensus.cc:348] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:28.080458 16103 raft_consensus.cc:374] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:28.080554 16103 raft_consensus.cc:729] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcfecb9143ff43deaff1fa75fd05df86, State: Initialized, Role: FOLLOWER
I20260807 19:37:28.080911 16103 consensus_queue.cc:260] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:28.081691 16103 ts_tablet_manager.cc:1428] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:37:28.081763 16122 tablet_replica.cc:439] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } } }
I20260807 19:37:28.082085 16122 tablet_replica.cc:442] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:28.088955 16102 tablet_bootstrap.cc:492] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1: No bootstrap required, opened a new log
I20260807 19:37:28.089157 16102 ts_tablet_manager.cc:1397] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.002s
I20260807 19:37:28.089553 16104 tablet_bootstrap.cc:492] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: No bootstrap required, opened a new log
I20260807 19:37:28.090021 16104 ts_tablet_manager.cc:1397] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.003s
I20260807 19:37:28.090005 16102 raft_consensus.cc:348] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:28.090261 16102 raft_consensus.cc:374] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:28.090349 16102 raft_consensus.cc:729] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4141b16354d84d0e9063d6c08721c9e1, State: Initialized, Role: FOLLOWER
I20260807 19:37:28.090638 16102 consensus_queue.cc:260] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:28.090798 16104 raft_consensus.cc:348] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:28.091104 16104 raft_consensus.cc:374] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:28.091233 16104 raft_consensus.cc:729] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c58eafe073884b1993bff2884c85a327, State: Initialized, Role: FOLLOWER
I20260807 19:37:28.091087 16109 tablet_replica.cc:439] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } } }
I20260807 19:37:28.091459 16109 tablet_replica.cc:442] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:28.091516 16104 consensus_queue.cc:260] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:28.091991 16102 ts_tablet_manager.cc:1428] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 19:37:28.091979 16110 tablet_replica.cc:439] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } } }
I20260807 19:37:28.092413 16110 tablet_replica.cc:442] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:28.093626 16104 ts_tablet_manager.cc:1428] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 19:37:28.114144 16110 raft_consensus.cc:482] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:28.114452 16110 raft_consensus.cc:504] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:28.116446 16110 leader_election.cc:288] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dcfecb9143ff43deaff1fa75fd05df86 (127.14.16.66:36363), 4141b16354d84d0e9063d6c08721c9e1 (127.14.16.67:35955)
I20260807 19:37:28.120091 16109 raft_consensus.cc:482] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:28.120316 16109 raft_consensus.cc:504] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:28.122130 16109 leader_election.cc:288] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c58eafe073884b1993bff2884c85a327 (127.14.16.65:34167), dcfecb9143ff43deaff1fa75fd05df86 (127.14.16.66:36363)
I20260807 19:37:28.131953 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.117583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:48979 (local address 127.14.16.67:35955)
0807 19:37:28.117824 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.117833 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.120873 (+ 3040us) server_negotiation.cc:407] Connection header received
0807 19:37:28.121156 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.121162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.121292 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.121368 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:28.122247 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.123476 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.128596 (+ 5120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.130093 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.130223 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.130313 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.131446 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.131598 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:37:28.132926 16097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.117105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:46745 (local address 127.14.16.66:36363)
0807 19:37:28.117281 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.117289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.120219 (+ 2930us) server_negotiation.cc:407] Connection header received
0807 19:37:28.120535 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.120541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.120629 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.120745 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:28.121608 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.122992 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.126583 (+ 3591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.128239 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.128372 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.128463 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.132400 (+ 3937us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.132563 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 19:37:28.133930 16147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.116880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:36363 (local address 127.14.16.65:46745)
0807 19:37:28.120088 (+ 3208us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:28.120106 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:28.120354 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:28.120956 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:28.120961 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:28.120983 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:28.121405 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.121421 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.123196 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.123201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.126201 (+ 3000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.126220 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.130529 (+ 4309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.130536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.130739 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.130778 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 19:37:28.133300 (+ 2522us) client_negotiation.cc:240] Negotiation successful
0807 19:37:28.133462 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2837,"mutex_wait_us":28,"thread_start_us":2920,"threads_started":1}
I20260807 19:37:28.135273 16101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.123326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:48975 (local address 127.14.16.65:34167)
0807 19:37:28.123495 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.123502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.126736 (+ 3234us) server_negotiation.cc:407] Connection header received
0807 19:37:28.126994 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.126999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.127081 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.127145 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:28.127957 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.129252 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.132056 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.133425 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.133569 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.133653 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.134842 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.134995 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:37:28.135102 15886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64e8d2af93842f2b4edb00df548ff31" candidate_uuid: "c58eafe073884b1993bff2884c85a327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfecb9143ff43deaff1fa75fd05df86" is_pre_election: true
I20260807 19:37:28.135640 15886 raft_consensus.cc:2454] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c58eafe073884b1993bff2884c85a327 in term 0.
I20260807 19:37:28.136283 16148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.122946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:34167 (local address 127.14.16.67:48975)
0807 19:37:28.126642 (+ 3696us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:28.126663 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:28.126844 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:28.127376 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:28.127381 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:28.127410 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:28.127769 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.127781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.129455 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.129460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.131841 (+ 2381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.131856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.134301 (+ 2445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.134308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.134518 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.134560 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 19:37:28.135662 (+ 1102us) client_negotiation.cc:240] Negotiation successful
0807 19:37:28.135931 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3559,"thread_start_us":3389,"threads_started":1}
I20260807 19:37:28.136356 15670 leader_election.cc:302] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c58eafe073884b1993bff2884c85a327, dcfecb9143ff43deaff1fa75fd05df86; no voters:
I20260807 19:37:28.136821 16110 raft_consensus.cc:2781] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:37:28.136957 16110 raft_consensus.cc:482] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:37:28.137065 16110 raft_consensus.cc:3037] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:28.140364 16110 raft_consensus.cc:504] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:28.141582 15886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64e8d2af93842f2b4edb00df548ff31" candidate_uuid: "c58eafe073884b1993bff2884c85a327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfecb9143ff43deaff1fa75fd05df86"
I20260807 19:37:28.141812 15886 raft_consensus.cc:3037] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:28.144587 16097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.126999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:53319 (local address 127.14.16.66:36363)
0807 19:37:28.133189 (+ 6190us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.133195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.136868 (+ 3673us) server_negotiation.cc:407] Connection header received
0807 19:37:28.137360 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.137365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.137463 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.137529 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:28.138089 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.139244 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.141516 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.143224 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.143404 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.143482 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.144127 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.144279 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6074,"threads_started":1}
I20260807 19:37:28.145289 16148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.127448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:36363 (local address 127.14.16.67:53319)
0807 19:37:28.136775 (+ 9327us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:28.136795 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:28.137162 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:28.137750 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:28.137755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:28.137781 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:28.137937 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.137946 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.139447 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.139452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.141347 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.141361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.143717 (+ 2356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.143723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.143934 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.143972 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:37:28.144865 (+ 893us) client_negotiation.cc:240] Negotiation successful
0807 19:37:28.145019 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9221,"threads_started":1}
I20260807 19:37:28.145470 15886 raft_consensus.cc:2454] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c58eafe073884b1993bff2884c85a327 in term 1.
I20260807 19:37:28.145915 16110 leader_election.cc:288] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [CANDIDATE]: Term 1 election: Requested vote from peers dcfecb9143ff43deaff1fa75fd05df86 (127.14.16.66:36363), 4141b16354d84d0e9063d6c08721c9e1 (127.14.16.67:35955)
I20260807 19:37:28.145915 15670 leader_election.cc:302] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c58eafe073884b1993bff2884c85a327, dcfecb9143ff43deaff1fa75fd05df86; no voters:
I20260807 19:37:28.146335 16110 raft_consensus.cc:2781] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:28.146986 16146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.117061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:35955 (local address 127.14.16.65:48979)
0807 19:37:28.120775 (+ 3714us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:28.120793 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:28.120966 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:28.121583 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:28.121588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:28.121615 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:28.122012 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.122025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.123728 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.123734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.126388 (+ 2654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.126402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.131031 (+ 4629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.131037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.131227 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.131258 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:37:28.132223 (+ 965us) client_negotiation.cc:240] Negotiation successful
0807 19:37:28.146304 (+ 14081us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3553,"mutex_wait_us":14104,"thread_start_us":2387,"threads_started":1}
I20260807 19:37:28.148065 16037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64e8d2af93842f2b4edb00df548ff31" candidate_uuid: "c58eafe073884b1993bff2884c85a327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4141b16354d84d0e9063d6c08721c9e1" is_pre_election: true
I20260807 19:37:28.148236 16036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64e8d2af93842f2b4edb00df548ff31" candidate_uuid: "c58eafe073884b1993bff2884c85a327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4141b16354d84d0e9063d6c08721c9e1"
I20260807 19:37:28.148388 16037 raft_consensus.cc:2454] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c58eafe073884b1993bff2884c85a327 in term 0.
I20260807 19:37:28.149071 16110 raft_consensus.cc:686] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [term 1 LEADER]: Becoming Leader. State: Replica: c58eafe073884b1993bff2884c85a327, State: Running, Role: LEADER
I20260807 19:37:28.149556 16110 consensus_queue.cc:237] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } }
I20260807 19:37:28.150789 15739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64e8d2af93842f2b4edb00df548ff31" candidate_uuid: "4141b16354d84d0e9063d6c08721c9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c58eafe073884b1993bff2884c85a327" is_pre_election: true
I20260807 19:37:28.153923 15886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64e8d2af93842f2b4edb00df548ff31" candidate_uuid: "4141b16354d84d0e9063d6c08721c9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfecb9143ff43deaff1fa75fd05df86" is_pre_election: true
I20260807 19:37:28.154330 15886 raft_consensus.cc:2379] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4141b16354d84d0e9063d6c08721c9e1 in current term 1: Already voted for candidate c58eafe073884b1993bff2884c85a327 in this term.
I20260807 19:37:28.155176 15961 leader_election.cc:302] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4141b16354d84d0e9063d6c08721c9e1; no voters: c58eafe073884b1993bff2884c85a327, dcfecb9143ff43deaff1fa75fd05df86
I20260807 19:37:28.155398 16151 tablet_replica.cc:439] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: TxnStatusTablet state changed. Reason: New leader c58eafe073884b1993bff2884c85a327. Latest consensus state: current_term: 1 leader_uuid: "c58eafe073884b1993bff2884c85a327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } } }
I20260807 19:37:28.155764 16151 tablet_replica.cc:442] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:37:28.156198 15571 catalog_manager.cc:5495] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 reported cstate change: term changed from 0 to 1, leader changed from <none> to c58eafe073884b1993bff2884c85a327 (127.14.16.65). New cstate: current_term: 1 leader_uuid: "c58eafe073884b1993bff2884c85a327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } health_report { overall_health: UNKNOWN } } }
I20260807 19:37:28.157478 16109 raft_consensus.cc:3037] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:28.161852 16109 raft_consensus.cc:2726] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:37:28.163265 16153 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 19:37:28.163594 16153 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 19:37:28.198453 16101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.190666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51756 (local address 127.14.16.65:34167)
0807 19:37:28.190860 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.190867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.191010 (+ 143us) server_negotiation.cc:407] Connection header received
0807 19:37:28.191356 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.191362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.191486 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.191640 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:28.192123 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.193318 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.195540 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.196229 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.196359 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.196961 (+ 602us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:28.197707 (+ 746us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:28.197865 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.198020 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.198125 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:37:28.198501 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.190601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:34167 (local address 127.0.0.1:51756)
0807 19:37:28.190889 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:28.190911 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:28.191080 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:28.191708 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:28.191714 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:28.191730 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:37:28.191957 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.191970 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.193484 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.193491 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.195347 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.195364 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.196414 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.196422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.196654 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.196745 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:37:28.197844 (+ 1099us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:37:28.197877 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:37:28.197984 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 19:37:28.198160 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 19:37:28.203449 16036 raft_consensus.cc:1261] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1 [term 1 FOLLOWER]: Refusing update from remote peer c58eafe073884b1993bff2884c85a327: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:28.203761 15886 raft_consensus.cc:1261] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86 [term 1 FOLLOWER]: Refusing update from remote peer c58eafe073884b1993bff2884c85a327: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:28.204325 16110 consensus_queue.cc:1035] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:28.204661 16151 consensus_queue.cc:1035] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:28.215049 16109 tablet_replica.cc:439] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1: TxnStatusTablet state changed. Reason: New leader c58eafe073884b1993bff2884c85a327. Latest consensus state: current_term: 1 leader_uuid: "c58eafe073884b1993bff2884c85a327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } } }
I20260807 19:37:28.215433 16109 tablet_replica.cc:442] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:28.219548 16110 tablet_replica.cc:439] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c58eafe073884b1993bff2884c85a327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } } }
I20260807 19:37:28.219870 16110 tablet_replica.cc:442] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:37:28.220522 16119 mvcc.cc:205] Tried to move back new op lower bound from 7315994411832512512 to 7315994411624710144. Current Snapshot: MvccSnapshot[applied={T|T < 7315994411832512512 or (T in {7315994411832512512})}]
I20260807 19:37:28.221228 16118 mvcc.cc:205] Tried to move back new op lower bound from 7315994411832512512 to 7315994411624710144. Current Snapshot: MvccSnapshot[applied={T|T < 7315994411832512512}]
I20260807 19:37:28.221670 16109 tablet_replica.cc:439] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c58eafe073884b1993bff2884c85a327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } } }
I20260807 19:37:28.221935 16109 tablet_replica.cc:442] T b64e8d2af93842f2b4edb00df548ff31 P 4141b16354d84d0e9063d6c08721c9e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:28.224617 16122 tablet_replica.cc:439] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86: TxnStatusTablet state changed. Reason: New leader c58eafe073884b1993bff2884c85a327. Latest consensus state: current_term: 1 leader_uuid: "c58eafe073884b1993bff2884c85a327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } } }
I20260807 19:37:28.225013 16122 tablet_replica.cc:442] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:28.226179 16151 tablet_replica.cc:439] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c58eafe073884b1993bff2884c85a327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } } }
I20260807 19:37:28.226449 16151 tablet_replica.cc:442] T b64e8d2af93842f2b4edb00df548ff31 P c58eafe073884b1993bff2884c85a327: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:37:28.229096 16122 tablet_replica.cc:439] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c58eafe073884b1993bff2884c85a327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 } } peers { permanent_uuid: "dcfecb9143ff43deaff1fa75fd05df86" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 36363 } } peers { permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 } } }
I20260807 19:37:28.229422 16122 tablet_replica.cc:442] T b64e8d2af93842f2b4edb00df548ff31 P dcfecb9143ff43deaff1fa75fd05df86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:28.246433 16101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.240151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:39759 (local address 127.14.16.65:34167)
0807 19:37:28.240338 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.240345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.241277 (+ 932us) server_negotiation.cc:407] Connection header received
0807 19:37:28.241484 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.241489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.241572 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.241671 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:28.242225 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.242986 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.244534 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.245069 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.245221 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.245656 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:28.245776 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:28.245872 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.246012 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.246094 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:37:28.246559 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.240228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:34167 (local address 127.14.16.66:39759)
0807 19:37:28.241161 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:28.241181 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:28.241349 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:28.241688 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:28.241693 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:28.241709 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:37:28.242063 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.242072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.243137 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.243142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.244392 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.244404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.245269 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.245278 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.245410 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.245468 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:37:28.245875 (+ 407us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:37:28.245890 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 19:37:28.245990 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 19:37:28.246149 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"thread_start_us":586,"threads_started":1}
I20260807 19:37:28.257893 15739 raft_consensus.cc:1261] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327 [term 1 FOLLOWER]: Refusing update from remote peer dcfecb9143ff43deaff1fa75fd05df86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:28.257992 16037 raft_consensus.cc:1261] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1 [term 1 FOLLOWER]: Refusing update from remote peer dcfecb9143ff43deaff1fa75fd05df86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:28.258618 16122 consensus_queue.cc:1035] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4141b16354d84d0e9063d6c08721c9e1" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 35955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:28.259193 16122 consensus_queue.cc:1035] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c58eafe073884b1993bff2884c85a327" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:28.333462 16149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.325820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:60269 (local address 127.14.16.66:36363)
0807 19:37:28.326022 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.326028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.326160 (+ 132us) server_negotiation.cc:407] Connection header received
0807 19:37:28.326699 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.326704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.326798 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.326876 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:28.327892 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.329018 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.330573 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.331045 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.331212 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.332315 (+ 1103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:28.332447 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:28.332508 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.333107 (+ 599us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.333201 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:37:28.334437 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.325275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:36363 (local address 127.14.16.65:60269)
0807 19:37:28.326052 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:28.326074 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:28.326488 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:28.327170 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:28.327174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:28.327188 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:37:28.327692 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.327705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.329177 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.329181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.330417 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:28.330429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.331399 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.331406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:28.331600 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.332030 (+ 430us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:37:28.332932 (+ 902us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:37:28.332941 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 19:37:28.333910 (+ 969us) client_negotiation.cc:240] Negotiation successful
0807 19:37:28.334080 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":761,"threads_started":1}
I20260807 19:37:28.392239 15571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:28.392509 15571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:28.395499 15571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 089fd72535ba4d9db54a3155acf7303d: Sending DeleteTablet for 3 replicas of tablet 9aff0bb9ef494a4a84ca62d5ef4b9318
I20260807 19:37:28.396471 16013 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aff0bb9ef494a4a84ca62d5ef4b9318 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:28 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:54158
I20260807 19:37:28.396690 15865 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aff0bb9ef494a4a84ca62d5ef4b9318 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:28 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:38004
I20260807 19:37:28.396823 16127 tablet_replica.cc:330] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1: stopping tablet replica
I20260807 19:37:28.396968 16129 tablet_replica.cc:330] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86: stopping tablet replica
I20260807 19:37:28.397053 16127 raft_consensus.cc:2229] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:28.397198 16129 raft_consensus.cc:2229] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:37:28.397287 16127 raft_consensus.cc:2258] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:28.397400 15718 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aff0bb9ef494a4a84ca62d5ef4b9318 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:28 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:51742
I20260807 19:37:28.397575 16129 raft_consensus.cc:2258] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:28.397691 16128 tablet_replica.cc:330] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327: stopping tablet replica
I20260807 19:37:28.397895 16128 raft_consensus.cc:2229] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:28.398099 16128 raft_consensus.cc:2258] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:28.398732 16127 ts_tablet_manager.cc:1905] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:28.400573 16128 ts_tablet_manager.cc:1905] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:28.404695 16129 ts_tablet_manager.cc:1905] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:28.427374 16127 ts_tablet_manager.cc:1918] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:37:28.430258 16128 ts_tablet_manager.cc:1918] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:37:28.433180 16129 ts_tablet_manager.cc:1918] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:37:28.433561 16127 log.cc:1196] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/9aff0bb9ef494a4a84ca62d5ef4b9318
I20260807 19:37:28.434254 16127 ts_tablet_manager.cc:1939] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P 4141b16354d84d0e9063d6c08721c9e1: Deleting consensus metadata
I20260807 19:37:28.434538 16128 log.cc:1196] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/9aff0bb9ef494a4a84ca62d5ef4b9318
I20260807 19:37:28.435235 16128 ts_tablet_manager.cc:1939] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P c58eafe073884b1993bff2884c85a327: Deleting consensus metadata
I20260807 19:37:28.436949 15558 catalog_manager.cc:4841] TS 4141b16354d84d0e9063d6c08721c9e1 (127.14.16.67:35955): tablet 9aff0bb9ef494a4a84ca62d5ef4b9318 (table test-table [id=febc68ab29bd4e4ea7e5172cd3c7b773]) successfully deleted
I20260807 19:37:28.437459 16129 log.cc:1196] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/9aff0bb9ef494a4a84ca62d5ef4b9318
I20260807 19:37:28.438179 16129 ts_tablet_manager.cc:1939] T 9aff0bb9ef494a4a84ca62d5ef4b9318 P dcfecb9143ff43deaff1fa75fd05df86: Deleting consensus metadata
I20260807 19:37:28.439898 15560 catalog_manager.cc:4841] TS c58eafe073884b1993bff2884c85a327 (127.14.16.65:34167): tablet 9aff0bb9ef494a4a84ca62d5ef4b9318 (table test-table [id=febc68ab29bd4e4ea7e5172cd3c7b773]) successfully deleted
I20260807 19:37:28.441601 15560 catalog_manager.cc:4841] TS dcfecb9143ff43deaff1fa75fd05df86 (127.14.16.66:36363): tablet 9aff0bb9ef494a4a84ca62d5ef4b9318 (table test-table [id=febc68ab29bd4e4ea7e5172cd3c7b773]) successfully deleted
Aug 07 19:37:28 dist-test-slave-07c2 krb5kdc[15526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:37:28.459852 16101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.409574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60856 (local address 127.14.16.65:34167)
0807 19:37:28.409752 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.409758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.442801 (+ 33043us) server_negotiation.cc:407] Connection header received
0807 19:37:28.443046 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.443052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.443136 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.443244 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:28.444755 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.445478 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.446717 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.447143 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.447281 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.451278 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:28.451345 (+ 67us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:28.451363 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:28.451410 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:28.455491 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:28.456437 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:28.456447 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:28.456451 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:28.456651 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:28.457168 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:28.457173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:28.457176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:28.457741 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:28.459016 (+ 1275us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.459171 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.459449 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 19:37:28.461184 15786 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60856
I20260807 19:37:28.516541 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.480624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.14.16.126:34425)
0807 19:37:28.480849 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.480857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.499891 (+ 19034us) server_negotiation.cc:407] Connection header received
0807 19:37:28.500254 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.500261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.500350 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.500425 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:28.502526 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.503701 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.505787 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.506541 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.506679 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.509910 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:28.509917 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:28.509922 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:28.509962 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:28.513487 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:28.514385 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:28.514391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:28.514395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:28.514536 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:28.515008 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:28.515012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:28.515014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:28.515430 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:28.515756 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.516148 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.516326 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 19:37:28.517539 15571 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36292
I20260807 19:37:28.519490 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 15650
I20260807 19:37:28.534319 16101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.520676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60868 (local address 127.14.16.65:34167)
0807 19:37:28.520913 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.520919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.521150 (+ 231us) server_negotiation.cc:407] Connection header received
0807 19:37:28.521349 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.521353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.521415 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.521518 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:28.522651 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.523501 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.524692 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.525213 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.525356 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.527877 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:28.527885 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:28.527895 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:28.527941 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:28.531523 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:28.532198 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:28.532204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:28.532208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:28.532398 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:28.532760 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:28.532765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:28.532768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:28.533143 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:28.533421 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.533670 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.533916 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 19:37:28.535032 15786 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60868
W20260807 19:37:28.535732 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 15650: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:37:28.535949 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 15650
I20260807 19:37:28.559361 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 15796
Aug 07 19:37:28 dist-test-slave-07c2 krb5kdc[15526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:37:28.574220 16149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.560148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42296 (local address 127.14.16.66:36363)
0807 19:37:28.560342 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.560351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.560435 (+ 84us) server_negotiation.cc:407] Connection header received
0807 19:37:28.560639 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.560645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.560721 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.560844 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:28.562047 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.562766 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.564098 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.564528 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.564664 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.568784 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:28.568798 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:28.568814 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:28.568853 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:28.571705 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:28.572437 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:28.572444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:28.572446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:28.572592 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:28.572902 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:28.572905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:28.572907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:28.573230 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:28.573461 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.573676 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.573841 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 19:37:28.574928 15928 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42296
W20260807 19:37:28.575544 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 15796: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:37:28.575740 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 15796
I20260807 19:37:28.601159 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 15942
Aug 07 19:37:28 dist-test-slave-07c2 krb5kdc[15526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:37:28.616200 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:28.601961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46074 (local address 127.14.16.67:35955)
0807 19:37:28.602134 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:28.602139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:28.602157 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:37:28.602309 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:28.602313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:28.602378 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:28.602464 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:28.603873 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.604644 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.605987 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:28.606529 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:28.606630 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:28.610784 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:28.610799 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:28.610816 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:28.610858 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:28.613563 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:28.614309 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:28.614316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:28.614318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:28.614452 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:28.614780 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:28.614783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:28.614785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:28.615204 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:28.615502 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:28.615717 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 19:37:28.615881 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 19:37:28.616880 16077 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46074
W20260807 19:37:28.617471 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 15942: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:37:28.617601 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 15942
I20260807 19:37:28.637351 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 15539
I20260807 19:37:28.638468 15616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:28.757453 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 15539
2026-08-07T19:37:28Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4747 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:37:28 dist-test-slave-07c2 krb5kdc[16175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:37:28 dist-test-slave-07c2 krb5kdc[16175](info): set up 2 sockets
Aug 07 19:37:28 dist-test-slave-07c2 krb5kdc[16175](info): commencing operation
krb5kdc: starting...
W20260807 19:37:29.873896 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:37:30 dist-test-slave-07c2 krb5kdc[16175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:37:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:37:30.079269 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:37435
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:37435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:30.400796 16188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:30.401120 16188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:30.401292 16188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:30.408674 16188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:30.408807 16188 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 19:37:30.408903 16188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:30.409000 16188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:30.409094 16188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:30.409184 16188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:30.419070 16188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:37435
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:37435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:30.420378 16188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:30.421702 16188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:30.433477 16193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:30.433521 16194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:30.433557 16196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:30.434294 16188 server_base.cc:1034] running on GCE node
I20260807 19:37:30.434970 16188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:30.436306 16188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:30.437502 16188 hybrid_clock.cc:648] HybridClock initialized: now 1786131450437426 us; error 30 us; skew 500 ppm
I20260807 19:37:30.437862 16188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:30 dist-test-slave-07c2 krb5kdc[16175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131450, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:30.440960 16188 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:37:30.442312 16188 webserver.cc:460] Webserver started at http://127.14.16.126:35071/ using document root <none> and password file <none>
I20260807 19:37:30.443008 16188 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:30.443122 16188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:30.443522 16188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:30.446409 16188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "8723cbb8088b42a8ba1a9ba6dc3be206"
format_stamp: "Formatted at 2026-08-07 19:37:30 on dist-test-slave-07c2"
server_key: "ecff4d25e701728e702bae6a805382b5"
server_key_iv: "46fb368852c5aa42c00638a2b29901ba"
server_key_version: "encryptionkey@0"
I20260807 19:37:30.447191 16188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "8723cbb8088b42a8ba1a9ba6dc3be206"
format_stamp: "Formatted at 2026-08-07 19:37:30 on dist-test-slave-07c2"
server_key: "ecff4d25e701728e702bae6a805382b5"
server_key_iv: "46fb368852c5aa42c00638a2b29901ba"
server_key_version: "encryptionkey@0"
I20260807 19:37:30.452301 16188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:37:30.455612 16203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:30.456853 16188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:37:30.457034 16188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "8723cbb8088b42a8ba1a9ba6dc3be206"
format_stamp: "Formatted at 2026-08-07 19:37:30 on dist-test-slave-07c2"
server_key: "ecff4d25e701728e702bae6a805382b5"
server_key_iv: "46fb368852c5aa42c00638a2b29901ba"
server_key_version: "encryptionkey@0"
I20260807 19:37:30.457245 16188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:30.475414 16188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:30.479656 16188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:30.479971 16188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:30.502036 16188 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:37435
I20260807 19:37:30.502246 16268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:37435 every 8 connection(s)
I20260807 19:37:30.503505 16188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:30.507117 16269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:30.509255 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 16188
I20260807 19:37:30.509474 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:30.510285 14401 external_mini_cluster.cc:1428] Setting key c6d5670fcd2b58a45a018440aa79a89f
I20260807 19:37:30.519872 16269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206: Bootstrap starting.
I20260807 19:37:30.523887 16269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206: Neither blocks nor log segments found. Creating new log.
Aug 07 19:37:30 dist-test-slave-07c2 krb5kdc[16175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:30.525125 16269 log.cc:824] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:30.528808 16269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206: No bootstrap required, opened a new log
I20260807 19:37:30.535374 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:30.512403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44608 (local address 127.14.16.126:37435)
0807 19:37:30.513251 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:30.513265 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:30.513304 (+ 39us) server_negotiation.cc:407] Connection header received
0807 19:37:30.514950 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:30.514972 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:30.515395 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:30.515703 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:30.517484 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:30.518773 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:30.520530 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:30.521185 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:30.521347 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:30.526423 (+ 5076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:30.526431 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:30.526479 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:30.526518 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:30.530573 (+ 4055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:30.531378 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:30.531387 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:30.531401 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:30.531624 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:30.532007 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:30.532012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:30.532015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:30.532782 (+ 767us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:30.533090 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:30.533475 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 19:37:30.533911 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":419,"threads_started":1}
I20260807 19:37:30.539186 16269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8723cbb8088b42a8ba1a9ba6dc3be206" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 37435 } }
I20260807 19:37:30.539605 16269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:30.539682 16269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8723cbb8088b42a8ba1a9ba6dc3be206, State: Initialized, Role: FOLLOWER
I20260807 19:37:30.540236 16269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8723cbb8088b42a8ba1a9ba6dc3be206" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 37435 } }
I20260807 19:37:30.540449 16269 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:30.540550 16269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:30.540673 16269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:30.543083 16269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8723cbb8088b42a8ba1a9ba6dc3be206" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 37435 } }
I20260807 19:37:30.543627 16269 leader_election.cc:302] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8723cbb8088b42a8ba1a9ba6dc3be206; no voters:
I20260807 19:37:30.544174 16269 leader_election.cc:288] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:30.544324 16274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:30.545490 16274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [term 1 LEADER]: Becoming Leader. State: Replica: 8723cbb8088b42a8ba1a9ba6dc3be206, State: Running, Role: LEADER
I20260807 19:37:30.546271 16274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8723cbb8088b42a8ba1a9ba6dc3be206" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 37435 } }
I20260807 19:37:30.546545 16269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:30.549324 16275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8723cbb8088b42a8ba1a9ba6dc3be206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8723cbb8088b42a8ba1a9ba6dc3be206" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 37435 } } }
I20260807 19:37:30.549664 16275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:30.550365 16276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8723cbb8088b42a8ba1a9ba6dc3be206. Latest consensus state: current_term: 1 leader_uuid: "8723cbb8088b42a8ba1a9ba6dc3be206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8723cbb8088b42a8ba1a9ba6dc3be206" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 37435 } } }
I20260807 19:37:30.550829 16276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:30.551582 16280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:30.556418 16280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:30.570118 16280 catalog_manager.cc:1347] Generated new cluster ID: b0b1b18574f7473b93f234167b402982
I20260807 19:37:30.570293 16280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:30.591423 16280 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:30.592742 16280 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:30.609010 16280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206: Generated new TSK 0
I20260807 19:37:30.609843 16280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:37:30.663242 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:37435
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:30.985433 16297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:30.985703 16297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:30.985881 16297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:30.993024 16297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:30.993158 16297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:30.993324 16297 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 19:37:30.993407 16297 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:37:30.993538 16297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:37:31.003175 16297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.16.126:37435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:31.004463 16297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:31.005754 16297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:31.018939 16304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:31.019035 16306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:31.019048 16303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:31.019851 16297 server_base.cc:1034] running on GCE node
I20260807 19:37:31.020511 16297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:31.021504 16297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:31.022739 16297 hybrid_clock.cc:648] HybridClock initialized: now 1786131451022702 us; error 46 us; skew 500 ppm
I20260807 19:37:31.023056 16297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:31 dist-test-slave-07c2 krb5kdc[16175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131451, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:31.026137 16297 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:37:31.027587 16297 webserver.cc:460] Webserver started at http://127.14.16.65:40425/ using document root <none> and password file <none>
I20260807 19:37:31.028177 16297 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:31.028276 16297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:31.028527 16297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:31.031431 16297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "c73aa055374b435eba0c9a1c60702c6f"
format_stamp: "Formatted at 2026-08-07 19:37:31 on dist-test-slave-07c2"
server_key: "4bbfbcf0e33c96570151436be7426f9f"
server_key_iv: "fbce13de30ee11f4777c66d60caf2456"
server_key_version: "encryptionkey@0"
I20260807 19:37:31.032156 16297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "c73aa055374b435eba0c9a1c60702c6f"
format_stamp: "Formatted at 2026-08-07 19:37:31 on dist-test-slave-07c2"
server_key: "4bbfbcf0e33c96570151436be7426f9f"
server_key_iv: "fbce13de30ee11f4777c66d60caf2456"
server_key_version: "encryptionkey@0"
I20260807 19:37:31.037065 16297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:37:31.040326 16314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:31.041541 16297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:37:31.041718 16297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "c73aa055374b435eba0c9a1c60702c6f"
format_stamp: "Formatted at 2026-08-07 19:37:31 on dist-test-slave-07c2"
server_key: "4bbfbcf0e33c96570151436be7426f9f"
server_key_iv: "fbce13de30ee11f4777c66d60caf2456"
server_key_version: "encryptionkey@0"
I20260807 19:37:31.041914 16297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:31.060402 16297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:31.064476 16297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:31.064775 16297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:31.067497 16297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:31.067643 16297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:31.067836 16297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:31.067986 16297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:37:31 dist-test-slave-07c2 krb5kdc[16175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131451, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:31.088783 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:31.067844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:44643 (local address 127.14.16.126:37435)
0807 19:37:31.068361 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:31.068370 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:31.069499 (+ 1129us) server_negotiation.cc:407] Connection header received
0807 19:37:31.071539 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:31.071545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:31.071646 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:31.071796 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:31.073832 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.074871 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.077356 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.077780 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.077918 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:31.081782 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:31.081787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:31.081790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:31.081823 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:31.084796 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:31.085797 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:31.085803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:31.085806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:31.085971 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:31.086664 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:31.086671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:31.086674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:31.087006 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:31.087170 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:31.088159 (+ 989us) server_negotiation.cc:299] Negotiation successful
0807 19:37:31.088385 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":259,"threads_started":1}
I20260807 19:37:31.089866 16324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:31.068560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:37435 (local address 127.14.16.65:44643)
0807 19:37:31.069335 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:31.069371 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:31.071210 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:31.072015 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:31.072022 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:31.072726 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:31.073639 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:31.073676 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.075057 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.075068 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:31.077188 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:31.077201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.078016 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.078025 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:31.078247 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:31.078929 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:31.078950 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:31.081550 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:31.084962 (+ 3412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:31.084968 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:31.084980 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:31.085649 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:31.086129 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:31.086138 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:31.086145 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:31.086496 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:31.087149 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:31.087157 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:31.087713 (+ 556us) client_negotiation.cc:769] Sending connection context
0807 19:37:31.088075 (+ 362us) client_negotiation.cc:240] Negotiation successful
0807 19:37:31.088626 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":334,"threads_started":1}
I20260807 19:37:31.112371 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:31.096576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:45639 (local address 127.14.16.126:37435)
0807 19:37:31.096752 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:31.096757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:31.097321 (+ 564us) server_negotiation.cc:407] Connection header received
0807 19:37:31.097543 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:31.097548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:31.097633 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:31.097733 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:31.099346 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.100278 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.101585 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.102089 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.102235 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:31.104443 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:31.104450 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:31.104454 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:31.104495 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:31.108789 (+ 4294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:31.110020 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:31.110026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:31.110029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:31.110197 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:31.110655 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:31.110662 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:31.110665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:31.111108 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:31.111259 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:31.111756 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 19:37:31.112012 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:37:31.112486 16374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:31.096566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:37435 (local address 127.14.16.65:45639)
0807 19:37:31.097209 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:31.097225 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:31.097399 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:31.097945 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:31.097950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:31.098607 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:31.099099 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:31.099118 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.100430 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.100436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:31.101442 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:31.101453 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.102286 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.102293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:31.102486 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:31.103157 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:31.103174 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:31.104241 (+ 1067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:31.108949 (+ 4708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:31.108988 (+ 39us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:31.108995 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:31.109834 (+ 839us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:31.110341 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:31.110347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:31.110351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:31.110506 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:31.111268 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:31.111277 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:31.111496 (+ 219us) client_negotiation.cc:769] Sending connection context
0807 19:37:31.111647 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 19:37:31.112021 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":359,"threads_started":1}
I20260807 19:37:31.127478 16297 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:39899
I20260807 19:37:31.127516 16444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:39899 every 8 connection(s)
I20260807 19:37:31.128762 16297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:37:31.130990 16445 heartbeater.cc:347] Connected to a master server at 127.14.16.126:37435
I20260807 19:37:31.131188 16445 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:31.131861 16445 heartbeater.cc:511] Master 127.14.16.126:37435 requested a full tablet report, sending...
I20260807 19:37:31.133034 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 16297
I20260807 19:37:31.133246 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:37:31.133812 16220 ts_manager.cc:194] Registered new tserver with Master: c73aa055374b435eba0c9a1c60702c6f (127.14.16.65:39899)
I20260807 19:37:31.133875 14401 external_mini_cluster.cc:1428] Setting key 619596dac916bc7d2b7b6941cd6845b5
I20260807 19:37:31.135475 16220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:44643
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:37:31.178894 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:37435
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:31.504943 16449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:31.505208 16449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:31.505380 16449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:31.512682 16449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:31.512784 16449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:31.512934 16449 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 19:37:31.513016 16449 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:37:31.513155 16449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:37:31.522578 16449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.16.126:37435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:31.523808 16449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:31.525060 16449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:31.539008 16454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:31.538942 16457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:31.538985 16455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:31.539606 16449 server_base.cc:1034] running on GCE node
I20260807 19:37:31.540373 16449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:31.541368 16449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:31.542603 16449 hybrid_clock.cc:648] HybridClock initialized: now 1786131451542561 us; error 64 us; skew 500 ppm
I20260807 19:37:31.542937 16449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:31 dist-test-slave-07c2 krb5kdc[16175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131451, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:31.555104 16449 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:37:31.556768 16449 webserver.cc:460] Webserver started at http://127.14.16.66:45243/ using document root <none> and password file <none>
I20260807 19:37:31.557560 16449 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:31.557720 16449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:31.558058 16449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:31.561260 16449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "5878be99c7a94bee9b7ea0ac718e7d9f"
format_stamp: "Formatted at 2026-08-07 19:37:31 on dist-test-slave-07c2"
server_key: "5b87ecf5a77d513d302f30c846f4ee3c"
server_key_iv: "56e4fed962f7e13a52d025a743934b84"
server_key_version: "encryptionkey@0"
I20260807 19:37:31.562258 16449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "5878be99c7a94bee9b7ea0ac718e7d9f"
format_stamp: "Formatted at 2026-08-07 19:37:31 on dist-test-slave-07c2"
server_key: "5b87ecf5a77d513d302f30c846f4ee3c"
server_key_iv: "56e4fed962f7e13a52d025a743934b84"
server_key_version: "encryptionkey@0"
I20260807 19:37:31.567854 16449 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:37:31.571201 16465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:31.572849 16449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:37:31.573035 16449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "5878be99c7a94bee9b7ea0ac718e7d9f"
format_stamp: "Formatted at 2026-08-07 19:37:31 on dist-test-slave-07c2"
server_key: "5b87ecf5a77d513d302f30c846f4ee3c"
server_key_iv: "56e4fed962f7e13a52d025a743934b84"
server_key_version: "encryptionkey@0"
I20260807 19:37:31.573292 16449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:31.604094 16449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:31.608428 16449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:31.608754 16449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:31.611601 16449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:31.611757 16449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:31.611884 16449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:31.611981 16449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:37:31 dist-test-slave-07c2 krb5kdc[16175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131451, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:31.638239 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:31.612961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:37435 (local address 127.14.16.66:45441)
0807 19:37:31.613910 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:31.613959 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:31.615935 (+ 1976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:31.616846 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:31.616857 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:31.617681 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:31.618919 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:31.618940 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.620445 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.620448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:31.622619 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:31.622644 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.623759 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.623767 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:31.624127 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:31.625161 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:31.625206 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:31.629025 (+ 3819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:31.633128 (+ 4103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:31.633139 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:31.633164 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:31.633663 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:31.634153 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:31.634164 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:31.634170 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:31.634405 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:31.634978 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:31.634986 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:31.635448 (+ 462us) client_negotiation.cc:769] Sending connection context
0807 19:37:31.635788 (+ 340us) client_negotiation.cc:240] Negotiation successful
0807 19:37:31.636420 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":392,"threads_started":1}
I20260807 19:37:31.639036 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:31.612330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:45441 (local address 127.14.16.126:37435)
0807 19:37:31.612545 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:31.612552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:31.614090 (+ 1538us) server_negotiation.cc:407] Connection header received
0807 19:37:31.616263 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:31.616272 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:31.616374 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:31.616513 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:31.619110 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.620184 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.622880 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.623487 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.623636 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:31.629313 (+ 5677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:31.629322 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:31.629328 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:31.629384 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:31.632970 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:31.633805 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:31.633814 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:31.633819 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:31.633990 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:31.634530 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:31.634536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:31.634539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:31.634834 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:31.634989 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:31.638382 (+ 3393us) server_negotiation.cc:299] Negotiation successful
0807 19:37:31.638660 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:37:31.660418 16449 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:39403
I20260807 19:37:31.660506 16590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:39403 every 8 connection(s)
I20260807 19:37:31.661903 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:31.646003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:58981 (local address 127.14.16.126:37435)
0807 19:37:31.646216 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:31.646225 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:31.647170 (+ 945us) server_negotiation.cc:407] Connection header received
0807 19:37:31.647491 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:31.647497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:31.647595 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:31.647697 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:31.649273 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.650007 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.651814 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.652420 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.652554 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:31.655138 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:31.655148 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:31.655154 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:31.655205 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:31.658631 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:31.659338 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:31.659343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:31.659347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:31.659539 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:31.660159 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:31.660166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:31.660171 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:31.660717 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:31.660838 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:31.661410 (+ 572us) server_negotiation.cc:299] Negotiation successful
0807 19:37:31.661589 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:37:31.662009 16557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:31.646232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:37435 (local address 127.14.16.66:58981)
0807 19:37:31.647023 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:31.647047 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:31.647324 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:31.647906 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:31.647913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:31.648532 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:31.649115 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:31.649126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.650168 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.650176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:31.651598 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:31.651613 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:31.652638 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:31.652645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:31.652813 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:31.653516 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:31.653536 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:31.654906 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:31.658760 (+ 3854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:31.658767 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:31.658771 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:31.659193 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:31.659759 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:31.659767 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:31.659773 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:31.659968 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:31.660841 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:31.660846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:31.661050 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 19:37:31.661210 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 19:37:31.661541 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":408,"threads_started":1}
I20260807 19:37:31.662120 16449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:37:31.664652 16591 heartbeater.cc:347] Connected to a master server at 127.14.16.126:37435
I20260807 19:37:31.665207 16591 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:31.665989 16591 heartbeater.cc:511] Master 127.14.16.126:37435 requested a full tablet report, sending...
I20260807 19:37:31.667357 16220 ts_manager.cc:194] Registered new tserver with Master: 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403)
I20260807 19:37:31.668152 16220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:45441
I20260807 19:37:31.669298 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 16449
I20260807 19:37:31.669487 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:37:31.670030 14401 external_mini_cluster.cc:1428] Setting key 71adc6df8d577b171a051ae26cdec416
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:37:31.714279 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:37435
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:32.041579 16595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:32.041899 16595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:32.042079 16595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:32.049521 16595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:32.049676 16595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:32.049822 16595 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 19:37:32.049912 16595 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:37:32.050050 16595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:37:32.059674 16595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.16.126:37435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:32.060940 16595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:32.062248 16595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:32.075415 16601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:32.075585 16600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:32.075546 16603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:32.076416 16595 server_base.cc:1034] running on GCE node
I20260807 19:37:32.077145 16595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:32.078141 16595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:32.079394 16595 hybrid_clock.cc:648] HybridClock initialized: now 1786131452079331 us; error 50 us; skew 500 ppm
I20260807 19:37:32.079705 16595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:32 dist-test-slave-07c2 krb5kdc[16175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131452, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:32.082777 16595 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:37:32.084249 16595 webserver.cc:460] Webserver started at http://127.14.16.67:37651/ using document root <none> and password file <none>
I20260807 19:37:32.084918 16595 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:32.085068 16595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:32.085383 16595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:32.088325 16595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "bf677697f239490f9e29a661be949419"
format_stamp: "Formatted at 2026-08-07 19:37:32 on dist-test-slave-07c2"
server_key: "5fcd7549fcb29bcb5aa5a6c0baedeb24"
server_key_iv: "7c53f364b45cdc357aadc7f6a638c851"
server_key_version: "encryptionkey@0"
I20260807 19:37:32.089100 16595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "bf677697f239490f9e29a661be949419"
format_stamp: "Formatted at 2026-08-07 19:37:32 on dist-test-slave-07c2"
server_key: "5fcd7549fcb29bcb5aa5a6c0baedeb24"
server_key_iv: "7c53f364b45cdc357aadc7f6a638c851"
server_key_version: "encryptionkey@0"
I20260807 19:37:32.094430 16595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:37:32.097790 16610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:32.098984 16595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:37:32.099159 16595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "bf677697f239490f9e29a661be949419"
format_stamp: "Formatted at 2026-08-07 19:37:32 on dist-test-slave-07c2"
server_key: "5fcd7549fcb29bcb5aa5a6c0baedeb24"
server_key_iv: "7c53f364b45cdc357aadc7f6a638c851"
server_key_version: "encryptionkey@0"
I20260807 19:37:32.099419 16595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:32.116288 16595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:32.120689 16595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:32.121006 16595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:32.123823 16595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:32.123946 16595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:32.124073 16595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:32.124174 16595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:32.138135 16445 heartbeater.cc:503] Master 127.14.16.126:37435 was elected leader, sending a full tablet report...
Aug 07 19:37:32 dist-test-slave-07c2 krb5kdc[16175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131452, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:32.149062 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.124589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:60563 (local address 127.14.16.126:37435)
0807 19:37:32.124840 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.124847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.126134 (+ 1287us) server_negotiation.cc:407] Connection header received
0807 19:37:32.128998 (+ 2864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.129008 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.129109 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.129186 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:32.131683 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.132782 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.135496 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.136226 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.136404 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.141721 (+ 5317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:32.141727 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:32.141732 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:32.141775 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:32.145422 (+ 3647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:32.146331 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:32.146336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:32.146340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:32.146474 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:32.146988 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:32.146993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:32.146997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:32.147281 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:32.147486 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.148428 (+ 942us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.148682 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260807 19:37:32.150570 16619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.125100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:37435 (local address 127.14.16.67:60563)
0807 19:37:32.125968 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.126004 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.128637 (+ 2633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.129707 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.129718 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.130568 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:32.131464 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.131488 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.132975 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.132980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.135294 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.135339 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.136523 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.136533 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.136843 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.137715 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:32.137751 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:32.141429 (+ 3678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:32.145575 (+ 4146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:32.145583 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:32.145597 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:32.146186 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:32.146590 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:32.146595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:32.146599 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:32.146864 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:32.147458 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:32.147469 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:32.147945 (+ 476us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.148317 (+ 372us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.148894 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":383,"threads_started":1}
I20260807 19:37:32.173846 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.156838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:51829 (local address 127.14.16.126:37435)
0807 19:37:32.157003 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.157007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.157822 (+ 815us) server_negotiation.cc:407] Connection header received
0807 19:37:32.158086 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.158094 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.158195 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.158273 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:32.159928 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.160952 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.162926 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.163842 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.163995 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.166760 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:32.166765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:32.166768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:32.166801 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:32.170277 (+ 3476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:32.171407 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:32.171414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:32.171418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:32.171586 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:32.172111 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:32.172118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:32.172121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:32.172428 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:32.172624 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.173219 (+ 595us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.173466 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:37:32.173954 16690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.157099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:37435 (local address 127.14.16.67:51829)
0807 19:37:32.157725 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.157739 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.157900 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.158459 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.158462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.159131 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:32.159768 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.159779 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.161116 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.161121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.162780 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.162792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.164280 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.164288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.164532 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.165220 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:32.165247 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:32.166549 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:32.170421 (+ 3872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:32.170427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:32.170432 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:32.171193 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:32.171720 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:32.171739 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:32.171746 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:32.171937 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:32.172701 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:32.172711 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:32.172941 (+ 230us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.173126 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.173475 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":355,"threads_started":1}
I20260807 19:37:32.177675 16595 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:33787
I20260807 19:37:32.177757 16734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:33787 every 8 connection(s)
I20260807 19:37:32.179101 16595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:37:32.181023 16735 heartbeater.cc:347] Connected to a master server at 127.14.16.126:37435
I20260807 19:37:32.181252 16735 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:32.181867 16735 heartbeater.cc:511] Master 127.14.16.126:37435 requested a full tablet report, sending...
I20260807 19:37:32.183132 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 16595
I20260807 19:37:32.183123 16220 ts_manager.cc:194] Registered new tserver with Master: bf677697f239490f9e29a661be949419 (127.14.16.67:33787)
I20260807 19:37:32.183363 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:37:32.183830 14401 external_mini_cluster.cc:1428] Setting key 75e75f63d698b1e1708f8cea90c7c10e
I20260807 19:37:32.184092 16220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:60563
I20260807 19:37:32.186729 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:37:32 dist-test-slave-07c2 krb5kdc[16175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:37:32 dist-test-slave-07c2 krb5kdc[16175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:32.236557 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.223073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44616 (local address 127.14.16.126:37435)
0807 19:37:32.223387 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.223396 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.223611 (+ 215us) server_negotiation.cc:407] Connection header received
0807 19:37:32.223766 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.223771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.223838 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.223926 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:32.225292 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.225997 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.227144 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.227605 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.227699 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.231457 (+ 3758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:32.231463 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:32.231466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:32.231498 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:32.233916 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:32.234511 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:32.234514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:32.234517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:32.234631 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:32.234985 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:32.234991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:32.234994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:32.235347 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:32.235618 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.235920 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.236149 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 19:37:32.240581 16220 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:37:32.247823 16220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:37:32.270624 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.260365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38994 (local address 127.14.16.67:33787)
0807 19:37:32.261163 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.261173 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.261206 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:37:32.261451 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.261458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.261702 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.261901 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:32.262730 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.264616 (+ 1886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.267441 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.269021 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.269136 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.269416 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.270084 (+ 668us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.270265 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":394,"threads_started":1}
I20260807 19:37:32.271461 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.261368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39899 (local address 127.0.0.1:57222)
0807 19:37:32.261900 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.261925 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.262060 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.263839 (+ 1779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.263844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.263869 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:32.264307 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.264319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.266170 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.266176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.268398 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.268415 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.270529 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.270534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.270725 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.270766 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.270853 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.271056 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":585,"threads_started":1}
I20260807 19:37:32.271966 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.260139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:33787 (local address 127.0.0.1:38994)
0807 19:37:32.260965 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.261025 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.261209 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.261918 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.261926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.261958 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:32.262437 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.262465 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.264811 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.264819 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.267228 (+ 2409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.267244 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.269593 (+ 2349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.269599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.269807 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.269861 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.271186 (+ 1325us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.271338 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":361,"threads_started":1}
I20260807 19:37:32.272469 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.261756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57222 (local address 127.14.16.65:39899)
0807 19:37:32.262668 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.262678 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.262707 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:37:32.262803 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.262855 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.263235 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.263560 (+ 325us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:32.264486 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.265984 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.268610 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.270069 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.270192 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.270380 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.271997 (+ 1617us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.272183 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":674,"threads_started":1}
I20260807 19:37:32.277796 16662 tablet_service.cc:1467] Processing CreateTablet for tablet 4e4d036ae0794e31944fd44041bb5d20 (DEFAULT_TABLE table=test-table [id=bbf5315db8774febae67a8b46589d1f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:32.277776 16368 tablet_service.cc:1467] Processing CreateTablet for tablet 4e4d036ae0794e31944fd44041bb5d20 (DEFAULT_TABLE table=test-table [id=bbf5315db8774febae67a8b46589d1f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:32.279350 16368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e4d036ae0794e31944fd44041bb5d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:32.279358 16662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e4d036ae0794e31944fd44041bb5d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:32.280355 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.262291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:39403 (local address 127.0.0.1:59672)
0807 19:37:32.271711 (+ 9420us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.271730 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.271835 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.273525 (+ 1690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.273533 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.273578 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:32.273823 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.273834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.275457 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.275462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.277314 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.277329 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.279606 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.279612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.279799 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.279831 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.279935 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.280076 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9333,"thread_start_us":11420,"threads_started":1}
I20260807 19:37:32.280442 16753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.262564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59672 (local address 127.14.16.66:39403)
0807 19:37:32.273011 (+ 10447us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.273017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.273041 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:37:32.273128 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.273133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.273318 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.273477 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:32.273976 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.275244 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.277499 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.279401 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.279548 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.279738 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.280002 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.280161 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10321,"thread_start_us":10195,"threads_started":1}
I20260807 19:37:32.284363 16516 tablet_service.cc:1467] Processing CreateTablet for tablet 4e4d036ae0794e31944fd44041bb5d20 (DEFAULT_TABLE table=test-table [id=bbf5315db8774febae67a8b46589d1f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:32.285866 16516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e4d036ae0794e31944fd44041bb5d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:32.292722 16754 tablet_bootstrap.cc:492] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f: Bootstrap starting.
I20260807 19:37:32.294837 16755 tablet_bootstrap.cc:492] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419: Bootstrap starting.
I20260807 19:37:32.296542 16754 tablet_bootstrap.cc:654] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:32.298386 16755 tablet_bootstrap.cc:654] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:32.298494 16754 log.cc:824] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:32.299681 16755 log.cc:824] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:32.301977 16754 tablet_bootstrap.cc:492] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f: No bootstrap required, opened a new log
I20260807 19:37:32.302294 16754 ts_tablet_manager.cc:1397] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260807 19:37:32.302227 16757 tablet_bootstrap.cc:492] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f: Bootstrap starting.
I20260807 19:37:32.304746 16755 tablet_bootstrap.cc:492] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419: No bootstrap required, opened a new log
I20260807 19:37:32.305086 16755 ts_tablet_manager.cc:1397] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419: Time spent bootstrapping tablet: real 0.010s user 0.010s sys 0.000s
I20260807 19:37:32.305580 16757 tablet_bootstrap.cc:654] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:32.306473 16757 log.cc:824] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:32.308796 16757 tablet_bootstrap.cc:492] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f: No bootstrap required, opened a new log
I20260807 19:37:32.309043 16757 ts_tablet_manager.cc:1397] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.002s
I20260807 19:37:32.313604 16754 raft_consensus.cc:348] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } }
I20260807 19:37:32.314249 16754 raft_consensus.cc:374] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:32.314373 16754 raft_consensus.cc:729] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73aa055374b435eba0c9a1c60702c6f, State: Initialized, Role: FOLLOWER
I20260807 19:37:32.315048 16754 consensus_queue.cc:260] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } }
I20260807 19:37:32.315676 16755 raft_consensus.cc:348] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } }
I20260807 19:37:32.316169 16755 raft_consensus.cc:374] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:32.316318 16755 raft_consensus.cc:729] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf677697f239490f9e29a661be949419, State: Initialized, Role: FOLLOWER
I20260807 19:37:32.317113 16754 ts_tablet_manager.cc:1428] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260807 19:37:32.317034 16755 consensus_queue.cc:260] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } }
I20260807 19:37:32.317327 16757 raft_consensus.cc:348] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } }
I20260807 19:37:32.317796 16757 raft_consensus.cc:374] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:32.317965 16757 raft_consensus.cc:729] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5878be99c7a94bee9b7ea0ac718e7d9f, State: Initialized, Role: FOLLOWER
I20260807 19:37:32.318262 16735 heartbeater.cc:503] Master 127.14.16.126:37435 was elected leader, sending a full tablet report...
I20260807 19:37:32.318621 16757 consensus_queue.cc:260] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } }
I20260807 19:37:32.318898 16755 ts_tablet_manager.cc:1428] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260807 19:37:32.320185 16591 heartbeater.cc:503] Master 127.14.16.126:37435 was elected leader, sending a full tablet report...
I20260807 19:37:32.320771 16757 ts_tablet_manager.cc:1428] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f: Time spent starting tablet: real 0.012s user 0.005s sys 0.007s
W20260807 19:37:32.381167 16446 tablet.cc:2369] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:37:32.414430 16592 tablet.cc:2369] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:37:32.430954 16736 tablet.cc:2369] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:37:32.584578 16761 raft_consensus.cc:482] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:32.585004 16761 raft_consensus.cc:504] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } }
I20260807 19:37:32.586673 16761 leader_election.cc:288] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c73aa055374b435eba0c9a1c60702c6f (127.14.16.65:39899), 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403)
I20260807 19:37:32.593756 16619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.587054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39899 (local address 127.14.16.67:38077)
0807 19:37:32.587347 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.587369 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.587545 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.588037 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.588044 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.588082 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:32.588342 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.588356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.589779 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.589782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.591642 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.591657 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.593190 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.593194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.593341 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.593367 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.593444 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.593534 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":1}
I20260807 19:37:32.593930 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.587163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:38077 (local address 127.14.16.65:39899)
0807 19:37:32.587479 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.587486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.587511 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:37:32.587723 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.587732 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.587850 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.587990 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:32.588508 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.589638 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.591802 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.593029 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.593227 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.593311 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.593504 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.593656 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:37:32.594987 16753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.587646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:60665 (local address 127.14.16.66:39403)
0807 19:37:32.587843 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.587847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.588462 (+ 615us) server_negotiation.cc:407] Connection header received
0807 19:37:32.588640 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.588646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.588728 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.588840 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:32.589326 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.590271 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.592377 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.593822 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.594006 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.594089 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.594567 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.594709 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:37:32.595022 16763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.587499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:39403 (local address 127.14.16.67:60665)
0807 19:37:32.588320 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.588342 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.588465 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.588849 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.588852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.588871 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:32.589163 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.589170 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.590417 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.590423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.592217 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.592230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.594261 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.594267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.594387 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.594410 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.594527 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.594670 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":509,"threads_started":1}
I20260807 19:37:32.594864 16393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e4d036ae0794e31944fd44041bb5d20" candidate_uuid: "bf677697f239490f9e29a661be949419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73aa055374b435eba0c9a1c60702c6f" is_pre_election: true
I20260807 19:37:32.595464 16393 raft_consensus.cc:2454] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf677697f239490f9e29a661be949419 in term 0.
I20260807 19:37:32.596143 16614 leader_election.cc:302] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf677697f239490f9e29a661be949419, c73aa055374b435eba0c9a1c60702c6f; no voters:
I20260807 19:37:32.596073 16539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e4d036ae0794e31944fd44041bb5d20" candidate_uuid: "bf677697f239490f9e29a661be949419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" is_pre_election: true
I20260807 19:37:32.596550 16761 raft_consensus.cc:2781] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:37:32.596637 16539 raft_consensus.cc:2454] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf677697f239490f9e29a661be949419 in term 0.
I20260807 19:37:32.596740 16761 raft_consensus.cc:482] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:37:32.596848 16761 raft_consensus.cc:3037] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:32.599572 16761 raft_consensus.cc:504] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } }
I20260807 19:37:32.600260 16761 leader_election.cc:288] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [CANDIDATE]: Term 1 election: Requested vote from peers c73aa055374b435eba0c9a1c60702c6f (127.14.16.65:39899), 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403)
I20260807 19:37:32.600693 16393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e4d036ae0794e31944fd44041bb5d20" candidate_uuid: "bf677697f239490f9e29a661be949419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73aa055374b435eba0c9a1c60702c6f"
I20260807 19:37:32.600792 16539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e4d036ae0794e31944fd44041bb5d20" candidate_uuid: "bf677697f239490f9e29a661be949419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f"
I20260807 19:37:32.600916 16393 raft_consensus.cc:3037] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:32.601051 16539 raft_consensus.cc:3037] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:32.603578 16393 raft_consensus.cc:2454] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf677697f239490f9e29a661be949419 in term 1.
I20260807 19:37:32.604005 16539 raft_consensus.cc:2454] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf677697f239490f9e29a661be949419 in term 1.
I20260807 19:37:32.604029 16614 leader_election.cc:302] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf677697f239490f9e29a661be949419, c73aa055374b435eba0c9a1c60702c6f; no voters:
I20260807 19:37:32.604379 16761 raft_consensus.cc:2781] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:32.604843 16761 raft_consensus.cc:686] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 1 LEADER]: Becoming Leader. State: Replica: bf677697f239490f9e29a661be949419, State: Running, Role: LEADER
I20260807 19:37:32.605367 16761 consensus_queue.cc:237] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } }
I20260807 19:37:32.610222 16218 catalog_manager.cc:5495] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf677697f239490f9e29a661be949419 (127.14.16.67). New cstate: current_term: 1 leader_uuid: "bf677697f239490f9e29a661be949419" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } health_report { overall_health: UNKNOWN } } }
Aug 07 19:37:32 dist-test-slave-07c2 krb5kdc[16175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131450, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:32.680279 16771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.665710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:37435 (local address 127.0.0.1:44624)
0807 19:37:32.666401 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.666423 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.666573 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.666912 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.666920 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.667786 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:32.668158 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.668170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.669198 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.669204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.670691 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.670709 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.671648 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.671655 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.671829 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.672515 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:32.672541 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:32.675058 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:32.677766 (+ 2708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:32.677777 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:32.677782 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:32.678289 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:32.678626 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:32.678629 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:32.678631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:32.678770 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:32.679452 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:32.679458 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:32.679607 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.679741 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.679944 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":356,"threads_started":1}
I20260807 19:37:32.680357 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.665681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.14.16.126:37435)
0807 19:37:32.665886 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.665898 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.666518 (+ 620us) server_negotiation.cc:407] Connection header received
0807 19:37:32.666678 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.666682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.666756 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.666843 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:32.668315 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.669057 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.670949 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.671465 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.671562 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.675231 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:32.675236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:32.675240 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:32.675268 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:32.677644 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:32.678388 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:32.678391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:32.678394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:32.678516 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:32.678863 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:32.678866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:32.678868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:32.679317 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:32.679459 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.679840 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.680068 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":71}
I20260807 19:37:32.685169 16218 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:44624:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 19:37:32.686149 16218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:37:32.692624 16368 tablet_service.cc:1467] Processing CreateTablet for tablet f9b8e2c306c34fd3bf29645ba1c8783c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1501e404a3e9434f8592575a4329cb12]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:37:32.692623 16516 tablet_service.cc:1467] Processing CreateTablet for tablet f9b8e2c306c34fd3bf29645ba1c8783c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1501e404a3e9434f8592575a4329cb12]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:37:32.692821 16662 tablet_service.cc:1467] Processing CreateTablet for tablet f9b8e2c306c34fd3bf29645ba1c8783c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1501e404a3e9434f8592575a4329cb12]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:37:32.693610 16368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9b8e2c306c34fd3bf29645ba1c8783c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:32.693606 16516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9b8e2c306c34fd3bf29645ba1c8783c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:32.693640 16662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9b8e2c306c34fd3bf29645ba1c8783c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:32.698752 16755 tablet_bootstrap.cc:492] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419: Bootstrap starting.
I20260807 19:37:32.700205 16755 tablet_bootstrap.cc:654] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:32.700874 16757 tablet_bootstrap.cc:492] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f: Bootstrap starting.
I20260807 19:37:32.701434 16754 tablet_bootstrap.cc:492] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: Bootstrap starting.
I20260807 19:37:32.702858 16754 tablet_bootstrap.cc:654] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:32.703150 16757 tablet_bootstrap.cc:654] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:32.705411 16754 tablet_bootstrap.cc:492] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: No bootstrap required, opened a new log
I20260807 19:37:32.705569 16754 ts_tablet_manager.cc:1397] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 19:37:32.705718 16755 tablet_bootstrap.cc:492] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419: No bootstrap required, opened a new log
I20260807 19:37:32.705819 16757 tablet_bootstrap.cc:492] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f: No bootstrap required, opened a new log
I20260807 19:37:32.705888 16755 ts_tablet_manager.cc:1397] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 19:37:32.705968 16757 ts_tablet_manager.cc:1397] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260807 19:37:32.706405 16754 raft_consensus.cc:348] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:32.706663 16754 raft_consensus.cc:374] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:32.706537 16755 raft_consensus.cc:348] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:32.706784 16755 raft_consensus.cc:374] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:32.706785 16754 raft_consensus.cc:729] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73aa055374b435eba0c9a1c60702c6f, State: Initialized, Role: FOLLOWER
I20260807 19:37:32.706892 16755 raft_consensus.cc:729] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf677697f239490f9e29a661be949419, State: Initialized, Role: FOLLOWER
I20260807 19:37:32.706741 16757 raft_consensus.cc:348] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:32.706986 16757 raft_consensus.cc:374] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:32.707085 16757 raft_consensus.cc:729] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5878be99c7a94bee9b7ea0ac718e7d9f, State: Initialized, Role: FOLLOWER
I20260807 19:37:32.707024 16754 consensus_queue.cc:260] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:32.707127 16755 consensus_queue.cc:260] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:32.707387 16757 consensus_queue.cc:260] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:32.707726 16755 ts_tablet_manager.cc:1428] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:37:32.708019 16757 ts_tablet_manager.cc:1428] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:37:32.707878 16762 tablet_replica.cc:439] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } }
I20260807 19:37:32.708225 16754 ts_tablet_manager.cc:1428] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:37:32.708323 16762 tablet_replica.cc:442] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:32.708230 16761 tablet_replica.cc:439] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } }
I20260807 19:37:32.708469 16761 tablet_replica.cc:442] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:32.708411 16760 tablet_replica.cc:439] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } }
I20260807 19:37:32.708779 16760 tablet_replica.cc:442] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:32.820739 16760 raft_consensus.cc:482] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:32.821012 16760 raft_consensus.cc:504] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:32.822554 16760 leader_election.cc:288] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403), bf677697f239490f9e29a661be949419 (127.14.16.67:33787)
I20260807 19:37:32.829404 16753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.822984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:45791 (local address 127.14.16.66:39403)
0807 19:37:32.823165 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.823169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.823493 (+ 324us) server_negotiation.cc:407] Connection header received
0807 19:37:32.823770 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.823775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.823863 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.823997 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:32.824800 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.825843 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.827633 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.828401 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.828610 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.828688 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.828958 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.829098 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:37:32.829442 16775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.822824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:39403 (local address 127.14.16.65:45791)
0807 19:37:32.823383 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.823406 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.823546 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.824256 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.824263 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.824295 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:32.824649 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.824664 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.825981 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.825988 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.827466 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.827478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.828631 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.828637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.828795 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.828819 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.828930 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.829078 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":287,"threads_started":1}
I20260807 19:37:32.829787 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.823736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:57627 (local address 127.14.16.67:33787)
0807 19:37:32.824628 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.824632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.824675 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:37:32.824751 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.824765 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.824839 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.824945 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:32.825642 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.826329 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.828030 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.828815 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.828998 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.829078 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.829374 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.829511 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":788,"thread_start_us":433,"threads_started":1}
I20260807 19:37:32.829857 16776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.823539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:33787 (local address 127.14.16.65:57627)
0807 19:37:32.824224 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.824242 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.824380 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.824995 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.825000 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.825027 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:32.825400 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.825416 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.826475 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.826483 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.827856 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.827868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.829050 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.829056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.829213 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.829236 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.829342 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.829468 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":347,"threads_started":1}
I20260807 19:37:32.830415 16539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9b8e2c306c34fd3bf29645ba1c8783c" candidate_uuid: "c73aa055374b435eba0c9a1c60702c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" is_pre_election: true
I20260807 19:37:32.830713 16539 raft_consensus.cc:2454] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c73aa055374b435eba0c9a1c60702c6f in term 0.
I20260807 19:37:32.830747 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9b8e2c306c34fd3bf29645ba1c8783c" candidate_uuid: "c73aa055374b435eba0c9a1c60702c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf677697f239490f9e29a661be949419" is_pre_election: true
I20260807 19:37:32.831182 16688 raft_consensus.cc:2454] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c73aa055374b435eba0c9a1c60702c6f in term 0.
I20260807 19:37:32.831239 16318 leader_election.cc:302] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5878be99c7a94bee9b7ea0ac718e7d9f, c73aa055374b435eba0c9a1c60702c6f; no voters:
I20260807 19:37:32.831717 16760 raft_consensus.cc:2781] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:37:32.831849 16760 raft_consensus.cc:482] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:37:32.831955 16760 raft_consensus.cc:3037] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:32.834268 16760 raft_consensus.cc:504] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:32.834810 16760 leader_election.cc:288] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [CANDIDATE]: Term 1 election: Requested vote from peers 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403), bf677697f239490f9e29a661be949419 (127.14.16.67:33787)
I20260807 19:37:32.835300 16539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9b8e2c306c34fd3bf29645ba1c8783c" candidate_uuid: "c73aa055374b435eba0c9a1c60702c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f"
I20260807 19:37:32.835364 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9b8e2c306c34fd3bf29645ba1c8783c" candidate_uuid: "c73aa055374b435eba0c9a1c60702c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf677697f239490f9e29a661be949419"
I20260807 19:37:32.835536 16539 raft_consensus.cc:3037] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:32.835548 16688 raft_consensus.cc:3037] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:32.838507 16539 raft_consensus.cc:2454] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c73aa055374b435eba0c9a1c60702c6f in term 1.
I20260807 19:37:32.838523 16688 raft_consensus.cc:2454] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c73aa055374b435eba0c9a1c60702c6f in term 1.
I20260807 19:37:32.838913 16318 leader_election.cc:302] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5878be99c7a94bee9b7ea0ac718e7d9f, c73aa055374b435eba0c9a1c60702c6f; no voters:
I20260807 19:37:32.839185 16760 raft_consensus.cc:2781] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:32.839648 16760 raft_consensus.cc:686] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [term 1 LEADER]: Becoming Leader. State: Replica: c73aa055374b435eba0c9a1c60702c6f, State: Running, Role: LEADER
I20260807 19:37:32.840216 16760 consensus_queue.cc:237] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:32.843720 16778 tablet_replica.cc:439] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: TxnStatusTablet state changed. Reason: New leader c73aa055374b435eba0c9a1c60702c6f. Latest consensus state: current_term: 1 leader_uuid: "c73aa055374b435eba0c9a1c60702c6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } }
I20260807 19:37:32.844200 16778 tablet_replica.cc:442] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:37:32.845260 16780 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 19:37:32.845523 16780 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 19:37:32.845180 16220 catalog_manager.cc:5495] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f reported cstate change: term changed from 0 to 1, leader changed from <none> to c73aa055374b435eba0c9a1c60702c6f (127.14.16.65). New cstate: current_term: 1 leader_uuid: "c73aa055374b435eba0c9a1c60702c6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } health_report { overall_health: UNKNOWN } } }
I20260807 19:37:32.868965 16771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.862889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39899 (local address 127.0.0.1:57232)
0807 19:37:32.863134 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.863151 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.863277 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.863835 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.863839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.863848 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:37:32.863993 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.864001 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.865456 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.865461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.866422 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.866433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.867461 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.867466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.867613 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.867661 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:37:32.868572 (+ 911us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:37:32.868587 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.868637 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.868747 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 19:37:32.869092 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.862974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57232 (local address 127.14.16.65:39899)
0807 19:37:32.863156 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.863162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.863210 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:37:32.863505 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.863511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.863606 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.863802 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:32.864132 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.865312 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.866555 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.867227 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.867456 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.867837 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:32.868462 (+ 625us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:32.868612 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.868661 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.868744 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:37:32.875995 16688 raft_consensus.cc:1261] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Refusing update from remote peer c73aa055374b435eba0c9a1c60702c6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:32.876219 16539 raft_consensus.cc:1261] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Refusing update from remote peer c73aa055374b435eba0c9a1c60702c6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:32.876854 16778 consensus_queue.cc:1035] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:32.877174 16760 consensus_queue.cc:1035] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:32.885082 16762 tablet_replica.cc:439] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f: TxnStatusTablet state changed. Reason: New leader c73aa055374b435eba0c9a1c60702c6f. Latest consensus state: current_term: 1 leader_uuid: "c73aa055374b435eba0c9a1c60702c6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } }
I20260807 19:37:32.885404 16762 tablet_replica.cc:442] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:32.889331 16783 mvcc.cc:205] Tried to move back new op lower bound from 7315994430971265024 to 7315994430837837824. Current Snapshot: MvccSnapshot[applied={T|T < 7315994430971265024}]
I20260807 19:37:32.889914 16778 tablet_replica.cc:439] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c73aa055374b435eba0c9a1c60702c6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } }
I20260807 19:37:32.890888 16778 tablet_replica.cc:442] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:37:32.899410 16761 tablet_replica.cc:439] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419: TxnStatusTablet state changed. Reason: New leader c73aa055374b435eba0c9a1c60702c6f. Latest consensus state: current_term: 1 leader_uuid: "c73aa055374b435eba0c9a1c60702c6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } }
I20260807 19:37:32.899757 16761 tablet_replica.cc:442] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:32.900110 16778 tablet_replica.cc:439] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c73aa055374b435eba0c9a1c60702c6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } }
I20260807 19:37:32.900398 16778 tablet_replica.cc:442] T f9b8e2c306c34fd3bf29645ba1c8783c P c73aa055374b435eba0c9a1c60702c6f: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:37:32.902899 16762 tablet_replica.cc:439] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c73aa055374b435eba0c9a1c60702c6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } }
I20260807 19:37:32.903162 16762 tablet_replica.cc:442] T f9b8e2c306c34fd3bf29645ba1c8783c P 5878be99c7a94bee9b7ea0ac718e7d9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:32.908828 16761 tablet_replica.cc:439] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c73aa055374b435eba0c9a1c60702c6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } } }
I20260807 19:37:32.909111 16761 tablet_replica.cc:442] T f9b8e2c306c34fd3bf29645ba1c8783c P bf677697f239490f9e29a661be949419: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:37:32.915549 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.908713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39004 (local address 127.14.16.67:33787)
0807 19:37:32.908922 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.908928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.910189 (+ 1261us) server_negotiation.cc:407] Connection header received
0807 19:37:32.910503 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.910508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.910604 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.910721 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:32.911326 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.912047 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.913506 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.913907 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.914045 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.914449 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:32.914931 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:32.915085 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.915185 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.915274 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:37:32.931537 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.925534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:38837 (local address 127.14.16.65:39899)
0807 19:37:32.925793 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.925799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.926129 (+ 330us) server_negotiation.cc:407] Connection header received
0807 19:37:32.926315 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.926319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.926384 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.926457 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:32.927028 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.927841 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.929842 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.930268 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.930358 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.930866 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:32.930967 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:32.931031 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.931195 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.931270 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:37:32.931725 16796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.925312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39899 (local address 127.14.16.67:38837)
0807 19:37:32.926017 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.926036 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.926186 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.926503 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.926509 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.926525 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:37:32.926872 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.926884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.927968 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.927974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.929711 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.929726 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.930441 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.930448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.930627 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.930694 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:37:32.931071 (+ 377us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:37:32.931085 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.931169 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.931336 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":423,"threads_started":1}
I20260807 19:37:32.941278 16539 raft_consensus.cc:1261] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Refusing update from remote peer bf677697f239490f9e29a661be949419: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:32.941597 16393 raft_consensus.cc:1261] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Refusing update from remote peer bf677697f239490f9e29a661be949419: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:32.942042 16764 consensus_queue.cc:1035] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:32.942477 16761 consensus_queue.cc:1035] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:32.948243 16785 mvcc.cc:205] Tried to move back new op lower bound from 7315994431242031104 to 7315994429876154368. Current Snapshot: MvccSnapshot[applied={T|T < 7315994431242031104}]
I20260807 19:37:32.951583 16784 mvcc.cc:205] Tried to move back new op lower bound from 7315994431242031104 to 7315994429876154368. Current Snapshot: MvccSnapshot[applied={T|T < 7315994431242031104}]
I20260807 19:37:32.982400 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.975168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:48397 (local address 127.14.16.67:33787)
0807 19:37:32.975392 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:32.975398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:32.976210 (+ 812us) server_negotiation.cc:407] Connection header received
0807 19:37:32.976440 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:32.976445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:32.976522 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:32.976611 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:32.977389 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.978413 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.980282 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.980905 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.981043 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.981748 (+ 705us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:32.981871 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:32.981946 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:32.982103 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 19:37:32.982164 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:37:32.982661 16802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:32.975075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:33787 (local address 127.14.16.65:48397)
0807 19:37:32.976091 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:32.976111 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:32.976290 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:32.976642 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:32.976646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:32.976658 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:37:32.977190 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.977206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.978621 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.978625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.980137 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:32.980148 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:32.981234 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:32.981240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:32.981443 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:32.981516 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:37:32.981978 (+ 462us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:37:32.981990 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:37:32.982093 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 19:37:32.982240 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"thread_start_us":612,"threads_started":1}
I20260807 19:37:33.020354 16218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:33.020691 16218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:33.024153 16218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206: Sending DeleteTablet for 3 replicas of tablet 4e4d036ae0794e31944fd44041bb5d20
I20260807 19:37:33.025357 16662 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e4d036ae0794e31944fd44041bb5d20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:33 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:38994
I20260807 19:37:33.025388 16516 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e4d036ae0794e31944fd44041bb5d20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:33 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:59672
I20260807 19:37:33.025493 16368 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e4d036ae0794e31944fd44041bb5d20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:33 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:57222
I20260807 19:37:33.027825 16218 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:37:33.028888 16218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:37:33.041754 16515 tablet_service.cc:1467] Processing CreateTablet for tablet 8459c193c41a4751aa4386fc34c880db (DEFAULT_TABLE table=test-table [id=d221c9c6be2e45579a138cfa41a8b91b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:33.042429 16515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8459c193c41a4751aa4386fc34c880db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:33.046314 16807 tablet_replica.cc:330] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f: stopping tablet replica
I20260807 19:37:33.046765 16807 raft_consensus.cc:2229] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:33.047279 16807 raft_consensus.cc:2258] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:33.047176 16661 tablet_service.cc:1467] Processing CreateTablet for tablet 8459c193c41a4751aa4386fc34c880db (DEFAULT_TABLE table=test-table [id=d221c9c6be2e45579a138cfa41a8b91b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:33.047799 16661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8459c193c41a4751aa4386fc34c880db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:33.049371 16807 ts_tablet_manager.cc:1905] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:33.056643 16806 tablet_replica.cc:330] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419: stopping tablet replica
I20260807 19:37:33.057052 16806 raft_consensus.cc:2229] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:37:33.057648 16757 tablet_bootstrap.cc:492] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f: Bootstrap starting.
I20260807 19:37:33.057718 16806 raft_consensus.cc:2258] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:33.058048 16755 tablet_bootstrap.cc:492] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419: Bootstrap starting.
I20260807 19:37:33.059779 16757 tablet_bootstrap.cc:654] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:33.059844 16755 tablet_bootstrap.cc:654] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:33.063356 16806 ts_tablet_manager.cc:1905] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:33.064147 16365 tablet_service.cc:1467] Processing CreateTablet for tablet 8459c193c41a4751aa4386fc34c880db (DEFAULT_TABLE table=test-table [id=d221c9c6be2e45579a138cfa41a8b91b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:33.065418 16365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8459c193c41a4751aa4386fc34c880db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:33.065598 16807 ts_tablet_manager.cc:1918] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:37:33.065773 16807 log.cc:1196] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/4e4d036ae0794e31944fd44041bb5d20
I20260807 19:37:33.066393 16807 ts_tablet_manager.cc:1939] T 4e4d036ae0794e31944fd44041bb5d20 P 5878be99c7a94bee9b7ea0ac718e7d9f: Deleting consensus metadata
I20260807 19:37:33.071449 16757 tablet_bootstrap.cc:492] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f: No bootstrap required, opened a new log
I20260807 19:37:33.071767 16757 ts_tablet_manager.cc:1397] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260807 19:37:33.072824 16757 raft_consensus.cc:348] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:33.073274 16757 raft_consensus.cc:374] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:33.073442 16754 tablet_bootstrap.cc:492] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f: Bootstrap starting.
I20260807 19:37:33.073422 16757 raft_consensus.cc:729] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5878be99c7a94bee9b7ea0ac718e7d9f, State: Initialized, Role: FOLLOWER
I20260807 19:37:33.073786 16757 consensus_queue.cc:260] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:33.075165 16754 tablet_bootstrap.cc:654] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:33.076514 16755 tablet_bootstrap.cc:492] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419: No bootstrap required, opened a new log
I20260807 19:37:33.076730 16755 ts_tablet_manager.cc:1397] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.000s
I20260807 19:37:33.076946 16757 ts_tablet_manager.cc:1428] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 19:37:33.077744 16755 raft_consensus.cc:348] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:33.078055 16755 raft_consensus.cc:374] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:33.078155 16755 raft_consensus.cc:729] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf677697f239490f9e29a661be949419, State: Initialized, Role: FOLLOWER
I20260807 19:37:33.078469 16755 consensus_queue.cc:260] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:33.079121 16755 ts_tablet_manager.cc:1428] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:37:33.080509 16207 catalog_manager.cc:4841] TS 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403): tablet 4e4d036ae0794e31944fd44041bb5d20 (table test-table [id=bbf5315db8774febae67a8b46589d1f3]) successfully deleted
I20260807 19:37:33.080770 16808 tablet_replica.cc:330] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f: stopping tablet replica
I20260807 19:37:33.081321 16808 raft_consensus.cc:2229] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:33.081751 16808 raft_consensus.cc:2258] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:33.083509 16806 ts_tablet_manager.cc:1918] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:37:33.083698 16806 log.cc:1196] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/4e4d036ae0794e31944fd44041bb5d20
I20260807 19:37:33.083765 16808 ts_tablet_manager.cc:1905] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:33.084188 16806 ts_tablet_manager.cc:1939] T 4e4d036ae0794e31944fd44041bb5d20 P bf677697f239490f9e29a661be949419: Deleting consensus metadata
I20260807 19:37:33.085474 16207 catalog_manager.cc:4841] TS bf677697f239490f9e29a661be949419 (127.14.16.67:33787): tablet 4e4d036ae0794e31944fd44041bb5d20 (table test-table [id=bbf5315db8774febae67a8b46589d1f3]) successfully deleted
I20260807 19:37:33.091990 16754 tablet_bootstrap.cc:492] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f: No bootstrap required, opened a new log
I20260807 19:37:33.092202 16754 ts_tablet_manager.cc:1397] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.004s
I20260807 19:37:33.092180 16808 ts_tablet_manager.cc:1918] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:37:33.092387 16808 log.cc:1196] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/4e4d036ae0794e31944fd44041bb5d20
I20260807 19:37:33.092978 16808 ts_tablet_manager.cc:1939] T 4e4d036ae0794e31944fd44041bb5d20 P c73aa055374b435eba0c9a1c60702c6f: Deleting consensus metadata
I20260807 19:37:33.093120 16754 raft_consensus.cc:348] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:33.093365 16754 raft_consensus.cc:374] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:33.093461 16754 raft_consensus.cc:729] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73aa055374b435eba0c9a1c60702c6f, State: Initialized, Role: FOLLOWER
I20260807 19:37:33.093699 16754 consensus_queue.cc:260] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:33.094431 16754 ts_tablet_manager.cc:1428] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:37:33.095118 16207 catalog_manager.cc:4841] TS c73aa055374b435eba0c9a1c60702c6f (127.14.16.65:39899): tablet 4e4d036ae0794e31944fd44041bb5d20 (table test-table [id=bbf5315db8774febae67a8b46589d1f3]) successfully deleted
I20260807 19:37:33.244730 16762 raft_consensus.cc:482] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:33.245003 16762 raft_consensus.cc:504] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:33.246938 16762 leader_election.cc:288] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c73aa055374b435eba0c9a1c60702c6f (127.14.16.65:39899), bf677697f239490f9e29a661be949419 (127.14.16.67:33787)
I20260807 19:37:33.256843 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:33.247474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:38563 (local address 127.14.16.65:39899)
0807 19:37:33.247719 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:33.247726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:33.248266 (+ 540us) server_negotiation.cc:407] Connection header received
0807 19:37:33.248642 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:33.248648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:33.248742 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:33.248822 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:33.249705 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.250928 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.253712 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.255402 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.255566 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:33.255653 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:33.256341 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 19:37:33.256507 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:37:33.257903 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:33.247182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39899 (local address 127.14.16.66:38563)
0807 19:37:33.248144 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:33.248165 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:33.248422 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:33.249041 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:33.249046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:33.249079 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:33.249473 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:33.249487 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.251136 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.251141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:33.253480 (+ 2339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:33.253497 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.255881 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.255888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:33.256113 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:33.256164 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 19:37:33.257360 (+ 1196us) client_negotiation.cc:240] Negotiation successful
0807 19:37:33.257531 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"thread_start_us":684,"threads_started":1}
I20260807 19:37:33.260520 16393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8459c193c41a4751aa4386fc34c880db" candidate_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73aa055374b435eba0c9a1c60702c6f" is_pre_election: true
I20260807 19:37:33.260855 16393 raft_consensus.cc:2454] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5878be99c7a94bee9b7ea0ac718e7d9f in term 0.
I20260807 19:37:33.261759 16469 leader_election.cc:302] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5878be99c7a94bee9b7ea0ac718e7d9f, c73aa055374b435eba0c9a1c60702c6f; no voters:
I20260807 19:37:33.262354 16762 raft_consensus.cc:2781] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:37:33.262485 16762 raft_consensus.cc:482] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:37:33.262598 16762 raft_consensus.cc:3037] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:33.265743 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:33.248671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:34625 (local address 127.14.16.67:33787)
0807 19:37:33.248878 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:33.248884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:33.258496 (+ 9612us) server_negotiation.cc:407] Connection header received
0807 19:37:33.258787 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:33.258793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:33.258876 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:33.258949 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:33.259608 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.260705 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.263089 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.264461 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.264605 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:33.264702 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:33.265308 (+ 606us) server_negotiation.cc:299] Negotiation successful
0807 19:37:33.265462 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:37:33.266413 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:33.248285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:33787 (local address 127.14.16.66:34625)
0807 19:37:33.258392 (+ 10107us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:33.258413 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:33.258610 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:33.259169 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:33.259174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:33.259204 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:33.259401 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:33.259415 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.260912 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.260917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:33.262851 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:33.262866 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.264859 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.264869 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:33.265094 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:33.265132 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:37:33.265944 (+ 812us) client_negotiation.cc:240] Negotiation successful
0807 19:37:33.266116 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9995,"thread_start_us":11411,"threads_started":1}
I20260807 19:37:33.266300 16762 raft_consensus.cc:504] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:33.267179 16762 leader_election.cc:288] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [CANDIDATE]: Term 1 election: Requested vote from peers c73aa055374b435eba0c9a1c60702c6f (127.14.16.65:39899), bf677697f239490f9e29a661be949419 (127.14.16.67:33787)
I20260807 19:37:33.267364 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8459c193c41a4751aa4386fc34c880db" candidate_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf677697f239490f9e29a661be949419" is_pre_election: true
I20260807 19:37:33.267684 16688 raft_consensus.cc:2454] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5878be99c7a94bee9b7ea0ac718e7d9f in term 0.
I20260807 19:37:33.267763 16393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8459c193c41a4751aa4386fc34c880db" candidate_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73aa055374b435eba0c9a1c60702c6f"
I20260807 19:37:33.267980 16393 raft_consensus.cc:3037] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:33.268096 16687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8459c193c41a4751aa4386fc34c880db" candidate_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf677697f239490f9e29a661be949419"
I20260807 19:37:33.268368 16687 raft_consensus.cc:3037] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:33.271595 16393 raft_consensus.cc:2454] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5878be99c7a94bee9b7ea0ac718e7d9f in term 1.
I20260807 19:37:33.272012 16687 raft_consensus.cc:2454] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5878be99c7a94bee9b7ea0ac718e7d9f in term 1.
I20260807 19:37:33.272204 16469 leader_election.cc:302] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5878be99c7a94bee9b7ea0ac718e7d9f, c73aa055374b435eba0c9a1c60702c6f; no voters:
I20260807 19:37:33.272562 16762 raft_consensus.cc:2781] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:33.273376 16762 raft_consensus.cc:686] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 LEADER]: Becoming Leader. State: Replica: 5878be99c7a94bee9b7ea0ac718e7d9f, State: Running, Role: LEADER
I20260807 19:37:33.273882 16762 consensus_queue.cc:237] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:33.279182 16218 catalog_manager.cc:5495] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66). New cstate: current_term: 1 leader_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } health_report { overall_health: UNKNOWN } } }
I20260807 19:37:33.309587 16753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:33.302557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59684 (local address 127.14.16.66:39403)
0807 19:37:33.302775 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:33.302781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:33.302800 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:37:33.302916 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:33.302922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:33.303052 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:33.303235 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:33.303720 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.304866 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.306830 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.307505 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.307648 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:33.308141 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:33.308931 (+ 790us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:33.309090 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:33.309207 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 19:37:33.309293 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:37:33.334013 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:33.326422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:51485 (local address 127.14.16.65:39899)
0807 19:37:33.326632 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:33.326638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:33.327305 (+ 667us) server_negotiation.cc:407] Connection header received
0807 19:37:33.327520 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:33.327528 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:33.327641 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:33.327763 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:33.328393 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.329563 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.331672 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.332518 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.332688 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:33.333226 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:33.333378 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:33.333471 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:33.333642 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 19:37:33.333725 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:37:33.334134 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:33.326251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39899 (local address 127.14.16.66:51485)
0807 19:37:33.327151 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:33.327172 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:33.327359 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:33.327790 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:33.327795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:33.327807 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:37:33.328198 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:33.328212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.329733 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.329739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:33.331468 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:33.331483 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.332762 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.332769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:33.332960 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:33.333027 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:37:33.333495 (+ 468us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:37:33.333509 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 19:37:33.333598 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 19:37:33.333755 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":540,"threads_started":1}
I20260807 19:37:33.346724 16687 raft_consensus.cc:1261] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Refusing update from remote peer 5878be99c7a94bee9b7ea0ac718e7d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:33.346794 16393 raft_consensus.cc:1261] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Refusing update from remote peer 5878be99c7a94bee9b7ea0ac718e7d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:33.347754 16814 consensus_queue.cc:1035] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:33.348567 16814 consensus_queue.cc:1035] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:33.398139 16802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:33.390976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:39403 (local address 127.14.16.65:47339)
0807 19:37:33.391176 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:33.391200 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:33.391348 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:33.392188 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:33.392193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:33.392210 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:37:33.392408 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:33.392420 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.393890 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.393896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:33.395571 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:33.395587 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.396773 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.396779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:33.397005 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:33.397062 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:37:33.397556 (+ 494us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:37:33.397569 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 19:37:33.397651 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 19:37:33.397808 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 19:37:33.398882 16753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:33.391464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:47339 (local address 127.14.16.66:39403)
0807 19:37:33.391648 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:33.391654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:33.391672 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:37:33.391761 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:33.391768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:33.391851 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:33.391947 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:33.392589 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.393718 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.395790 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:33.396507 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:33.396641 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:33.397272 (+ 631us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:33.397413 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:33.397498 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:33.398512 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0807 19:37:33.398606 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 19:37:33.400527 16784 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 19:37:35.002096 16218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:35.002373 16218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:35.006634 16218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206: Sending DeleteTablet for 3 replicas of tablet 8459c193c41a4751aa4386fc34c880db
I20260807 19:37:35.007648 16515 tablet_service.cc:1514] Processing DeleteTablet for tablet 8459c193c41a4751aa4386fc34c880db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:34 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:59672
I20260807 19:37:35.007669 16368 tablet_service.cc:1514] Processing DeleteTablet for tablet 8459c193c41a4751aa4386fc34c880db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:34 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:57222
I20260807 19:37:35.008140 16661 tablet_service.cc:1514] Processing DeleteTablet for tablet 8459c193c41a4751aa4386fc34c880db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:34 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:38994
I20260807 19:37:35.008387 16843 tablet_replica.cc:330] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f: stopping tablet replica
I20260807 19:37:35.008620 16843 raft_consensus.cc:2229] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:37:35.009097 16843 raft_consensus.cc:2258] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:35.009907 16843 ts_tablet_manager.cc:1905] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:35.015887 16843 ts_tablet_manager.cc:1918] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:37:35.016016 16843 log.cc:1196] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/8459c193c41a4751aa4386fc34c880db
I20260807 19:37:35.016470 16845 tablet_replica.cc:330] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419: stopping tablet replica
I20260807 19:37:35.016530 16843 ts_tablet_manager.cc:1939] T 8459c193c41a4751aa4386fc34c880db P 5878be99c7a94bee9b7ea0ac718e7d9f: Deleting consensus metadata
I20260807 19:37:35.016820 16845 raft_consensus.cc:2229] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:35.017130 16845 raft_consensus.cc:2258] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:35.017903 16207 catalog_manager.cc:4841] TS 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403): tablet 8459c193c41a4751aa4386fc34c880db (table test-table [id=d221c9c6be2e45579a138cfa41a8b91b]) successfully deleted
I20260807 19:37:35.018308 16845 ts_tablet_manager.cc:1905] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:35.019759 16844 tablet_replica.cc:330] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f: stopping tablet replica
I20260807 19:37:35.020010 16844 raft_consensus.cc:2229] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:35.020301 16844 raft_consensus.cc:2258] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:35.021241 16844 ts_tablet_manager.cc:1905] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:35.024534 16845 ts_tablet_manager.cc:1918] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:37:35.024663 16845 log.cc:1196] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/8459c193c41a4751aa4386fc34c880db
I20260807 19:37:35.025146 16845 ts_tablet_manager.cc:1939] T 8459c193c41a4751aa4386fc34c880db P bf677697f239490f9e29a661be949419: Deleting consensus metadata
I20260807 19:37:35.026304 16207 catalog_manager.cc:4841] TS bf677697f239490f9e29a661be949419 (127.14.16.67:33787): tablet 8459c193c41a4751aa4386fc34c880db (table test-table [id=d221c9c6be2e45579a138cfa41a8b91b]) successfully deleted
I20260807 19:37:35.027655 16844 ts_tablet_manager.cc:1918] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:37:35.027804 16844 log.cc:1196] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/8459c193c41a4751aa4386fc34c880db
I20260807 19:37:35.028267 16844 ts_tablet_manager.cc:1939] T 8459c193c41a4751aa4386fc34c880db P c73aa055374b435eba0c9a1c60702c6f: Deleting consensus metadata
I20260807 19:37:35.029218 16207 catalog_manager.cc:4841] TS c73aa055374b435eba0c9a1c60702c6f (127.14.16.65:39899): tablet 8459c193c41a4751aa4386fc34c880db (table test-table [id=d221c9c6be2e45579a138cfa41a8b91b]) successfully deleted
Aug 07 19:37:35 dist-test-slave-07c2 krb5kdc[16175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131455, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:37:35 dist-test-slave-07c2 krb5kdc[16175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131455, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:35.066524 16855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:35.052477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44636 (local address 127.14.16.126:37435)
0807 19:37:35.053059 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:35.053064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:35.053154 (+ 90us) server_negotiation.cc:407] Connection header received
0807 19:37:35.053370 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:35.053375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:35.053442 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:35.053542 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:35.054913 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:35.055781 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:35.056962 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:35.057367 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:35.057456 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:35.061300 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:35.061304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:35.061308 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:35.061339 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:35.064410 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:35.065007 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:35.065010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:35.065013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:35.065131 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:35.065426 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:35.065429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:35.065431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:35.065685 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:35.065855 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:35.066094 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 19:37:35.066254 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":286,"threads_started":1}
I20260807 19:37:35.070216 16218 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:37:35.070845 16218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:37:35.077343 16515 tablet_service.cc:1467] Processing CreateTablet for tablet 85bbca9d7fe145ebabbcd65fec03d068 (DEFAULT_TABLE table=test-table [id=e0a02db0b2b040c88f63bd5c1907c52f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:35.077353 16368 tablet_service.cc:1467] Processing CreateTablet for tablet 85bbca9d7fe145ebabbcd65fec03d068 (DEFAULT_TABLE table=test-table [id=e0a02db0b2b040c88f63bd5c1907c52f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:35.077816 16368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85bbca9d7fe145ebabbcd65fec03d068. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:35.077854 16515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85bbca9d7fe145ebabbcd65fec03d068. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:35.079846 16661 tablet_service.cc:1467] Processing CreateTablet for tablet 85bbca9d7fe145ebabbcd65fec03d068 (DEFAULT_TABLE table=test-table [id=e0a02db0b2b040c88f63bd5c1907c52f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:35.080272 16661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85bbca9d7fe145ebabbcd65fec03d068. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:35.083387 16856 tablet_bootstrap.cc:492] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f: Bootstrap starting.
I20260807 19:37:35.084211 16857 tablet_bootstrap.cc:492] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f: Bootstrap starting.
I20260807 19:37:35.085954 16857 tablet_bootstrap.cc:654] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:35.085954 16856 tablet_bootstrap.cc:654] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:35.086898 16858 tablet_bootstrap.cc:492] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419: Bootstrap starting.
I20260807 19:37:35.088405 16856 tablet_bootstrap.cc:492] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f: No bootstrap required, opened a new log
I20260807 19:37:35.088581 16856 ts_tablet_manager.cc:1397] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 19:37:35.088735 16857 tablet_bootstrap.cc:492] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f: No bootstrap required, opened a new log
I20260807 19:37:35.088922 16857 ts_tablet_manager.cc:1397] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 19:37:35.089125 16858 tablet_bootstrap.cc:654] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:35.089430 16856 raft_consensus.cc:348] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.089733 16856 raft_consensus.cc:374] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:35.089839 16856 raft_consensus.cc:729] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73aa055374b435eba0c9a1c60702c6f, State: Initialized, Role: FOLLOWER
I20260807 19:37:35.089756 16857 raft_consensus.cc:348] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.090008 16857 raft_consensus.cc:374] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:35.090135 16857 raft_consensus.cc:729] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5878be99c7a94bee9b7ea0ac718e7d9f, State: Initialized, Role: FOLLOWER
I20260807 19:37:35.090101 16856 consensus_queue.cc:260] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.090634 16856 ts_tablet_manager.cc:1428] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:37:35.090449 16857 consensus_queue.cc:260] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.091195 16857 ts_tablet_manager.cc:1428] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 19:37:35.093011 16858 tablet_bootstrap.cc:492] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419: No bootstrap required, opened a new log
I20260807 19:37:35.093165 16858 ts_tablet_manager.cc:1397] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260807 19:37:35.093973 16858 raft_consensus.cc:348] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.094290 16858 raft_consensus.cc:374] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:35.094398 16858 raft_consensus.cc:729] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf677697f239490f9e29a661be949419, State: Initialized, Role: FOLLOWER
I20260807 19:37:35.094655 16858 consensus_queue.cc:260] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.095593 16858 ts_tablet_manager.cc:1428] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419: Time spent starting tablet: real 0.002s user 0.002s sys 0.002s
I20260807 19:37:35.254827 16830 raft_consensus.cc:482] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:35.255044 16835 raft_consensus.cc:482] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:35.255064 16830 raft_consensus.cc:504] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.255244 16835 raft_consensus.cc:504] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.255846 16830 leader_election.cc:288] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403), bf677697f239490f9e29a661be949419 (127.14.16.67:33787)
I20260807 19:37:35.256027 16835 leader_election.cc:288] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c73aa055374b435eba0c9a1c60702c6f (127.14.16.65:39899), bf677697f239490f9e29a661be949419 (127.14.16.67:33787)
I20260807 19:37:35.256510 16539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85bbca9d7fe145ebabbcd65fec03d068" candidate_uuid: "c73aa055374b435eba0c9a1c60702c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" is_pre_election: true
I20260807 19:37:35.256515 16393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85bbca9d7fe145ebabbcd65fec03d068" candidate_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73aa055374b435eba0c9a1c60702c6f" is_pre_election: true
I20260807 19:37:35.256588 16687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85bbca9d7fe145ebabbcd65fec03d068" candidate_uuid: "c73aa055374b435eba0c9a1c60702c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf677697f239490f9e29a661be949419" is_pre_election: true
I20260807 19:37:35.256731 16539 raft_consensus.cc:2454] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c73aa055374b435eba0c9a1c60702c6f in term 0.
I20260807 19:37:35.256681 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85bbca9d7fe145ebabbcd65fec03d068" candidate_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf677697f239490f9e29a661be949419" is_pre_election: true
I20260807 19:37:35.256762 16393 raft_consensus.cc:2454] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5878be99c7a94bee9b7ea0ac718e7d9f in term 0.
I20260807 19:37:35.256848 16687 raft_consensus.cc:2454] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c73aa055374b435eba0c9a1c60702c6f in term 0.
I20260807 19:37:35.257375 16318 leader_election.cc:302] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf677697f239490f9e29a661be949419, c73aa055374b435eba0c9a1c60702c6f; no voters:
I20260807 19:37:35.257426 16469 leader_election.cc:302] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5878be99c7a94bee9b7ea0ac718e7d9f, c73aa055374b435eba0c9a1c60702c6f; no voters: bf677697f239490f9e29a661be949419
I20260807 19:37:35.257690 16835 raft_consensus.cc:2781] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:37:35.257740 16830 raft_consensus.cc:2781] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:37:35.257838 16835 raft_consensus.cc:482] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:37:35.257851 16830 raft_consensus.cc:482] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:37:35.257943 16835 raft_consensus.cc:3037] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:35.257943 16830 raft_consensus.cc:3037] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:35.260964 16830 raft_consensus.cc:504] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.260964 16835 raft_consensus.cc:504] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.261545 16835 leader_election.cc:288] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [CANDIDATE]: Term 1 election: Requested vote from peers c73aa055374b435eba0c9a1c60702c6f (127.14.16.65:39899), bf677697f239490f9e29a661be949419 (127.14.16.67:33787)
I20260807 19:37:35.261545 16830 leader_election.cc:288] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [CANDIDATE]: Term 1 election: Requested vote from peers 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403), bf677697f239490f9e29a661be949419 (127.14.16.67:33787)
I20260807 19:37:35.262125 16687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85bbca9d7fe145ebabbcd65fec03d068" candidate_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf677697f239490f9e29a661be949419"
I20260807 19:37:35.262091 16393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85bbca9d7fe145ebabbcd65fec03d068" candidate_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73aa055374b435eba0c9a1c60702c6f"
I20260807 19:37:35.262354 16687 raft_consensus.cc:3037] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:35.262303 16539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85bbca9d7fe145ebabbcd65fec03d068" candidate_uuid: "c73aa055374b435eba0c9a1c60702c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f"
I20260807 19:37:35.262416 16393 raft_consensus.cc:2379] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5878be99c7a94bee9b7ea0ac718e7d9f in current term 1: Already voted for candidate c73aa055374b435eba0c9a1c60702c6f in this term.
I20260807 19:37:35.262557 16539 raft_consensus.cc:2379] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c73aa055374b435eba0c9a1c60702c6f in current term 1: Already voted for candidate 5878be99c7a94bee9b7ea0ac718e7d9f in this term.
I20260807 19:37:35.262180 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85bbca9d7fe145ebabbcd65fec03d068" candidate_uuid: "c73aa055374b435eba0c9a1c60702c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf677697f239490f9e29a661be949419"
I20260807 19:37:35.263375 16318 leader_election.cc:302] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c73aa055374b435eba0c9a1c60702c6f; no voters: 5878be99c7a94bee9b7ea0ac718e7d9f, bf677697f239490f9e29a661be949419
I20260807 19:37:35.263653 16830 raft_consensus.cc:2726] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 19:37:35.264860 16687 raft_consensus.cc:2454] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5878be99c7a94bee9b7ea0ac718e7d9f in term 1.
I20260807 19:37:35.265226 16469 leader_election.cc:302] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5878be99c7a94bee9b7ea0ac718e7d9f, bf677697f239490f9e29a661be949419; no voters: c73aa055374b435eba0c9a1c60702c6f
I20260807 19:37:35.265509 16835 raft_consensus.cc:2781] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:35.265657 16835 raft_consensus.cc:686] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 LEADER]: Becoming Leader. State: Replica: 5878be99c7a94bee9b7ea0ac718e7d9f, State: Running, Role: LEADER
I20260807 19:37:35.265949 16835 consensus_queue.cc:237] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.268450 16220 catalog_manager.cc:5495] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66). New cstate: current_term: 1 leader_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } health_report { overall_health: UNKNOWN } } }
I20260807 19:37:35.333268 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:35.328570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59690 (local address 127.14.16.66:39403)
0807 19:37:35.329201 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:35.329205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:35.329224 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:37:35.329275 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:35.329278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:35.329336 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:35.329417 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:35.329746 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:35.330507 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:35.331773 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:35.332176 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:35.332266 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:35.332650 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:35.332752 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:35.332828 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:35.332960 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 19:37:35.333017 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":378,"threads_started":1}
I20260807 19:37:35.341464 16687 raft_consensus.cc:1261] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Refusing update from remote peer 5878be99c7a94bee9b7ea0ac718e7d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:35.341641 16393 raft_consensus.cc:1261] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Refusing update from remote peer 5878be99c7a94bee9b7ea0ac718e7d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:35.341976 16835 consensus_queue.cc:1035] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:35.342307 16814 consensus_queue.cc:1035] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:35.388669 16219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:35.388898 16219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:35.391754 16219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206: Sending DeleteTablet for 3 replicas of tablet 85bbca9d7fe145ebabbcd65fec03d068
I20260807 19:37:35.392488 16368 tablet_service.cc:1514] Processing DeleteTablet for tablet 85bbca9d7fe145ebabbcd65fec03d068 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:35 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:57222
I20260807 19:37:35.392575 16515 tablet_service.cc:1514] Processing DeleteTablet for tablet 85bbca9d7fe145ebabbcd65fec03d068 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:35 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:59672
I20260807 19:37:35.392625 16661 tablet_service.cc:1514] Processing DeleteTablet for tablet 85bbca9d7fe145ebabbcd65fec03d068 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:35 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:38994
I20260807 19:37:35.392802 16844 tablet_replica.cc:330] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f: stopping tablet replica
I20260807 19:37:35.392851 16843 tablet_replica.cc:330] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f: stopping tablet replica
I20260807 19:37:35.393000 16844 raft_consensus.cc:2229] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:35.393085 16845 tablet_replica.cc:330] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419: stopping tablet replica
I20260807 19:37:35.393047 16843 raft_consensus.cc:2229] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:37:35.393206 16844 raft_consensus.cc:2258] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:35.393260 16845 raft_consensus.cc:2229] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:35.393405 16843 raft_consensus.cc:2258] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:35.393432 16845 raft_consensus.cc:2258] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:35.393639 16219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:37:35.394295 16219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:37:35.394505 16843 ts_tablet_manager.cc:1905] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:35.394492 16845 ts_tablet_manager.cc:1905] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:35.394491 16844 ts_tablet_manager.cc:1905] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:35.400979 16843 ts_tablet_manager.cc:1918] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:37:35.401018 16844 ts_tablet_manager.cc:1918] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:37:35.401126 16843 log.cc:1196] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/85bbca9d7fe145ebabbcd65fec03d068
I20260807 19:37:35.401176 16844 log.cc:1196] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/85bbca9d7fe145ebabbcd65fec03d068
I20260807 19:37:35.401566 16843 ts_tablet_manager.cc:1939] T 85bbca9d7fe145ebabbcd65fec03d068 P 5878be99c7a94bee9b7ea0ac718e7d9f: Deleting consensus metadata
I20260807 19:37:35.401583 16845 ts_tablet_manager.cc:1918] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:37:35.401620 16844 ts_tablet_manager.cc:1939] T 85bbca9d7fe145ebabbcd65fec03d068 P c73aa055374b435eba0c9a1c60702c6f: Deleting consensus metadata
I20260807 19:37:35.401721 16845 log.cc:1196] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/85bbca9d7fe145ebabbcd65fec03d068
I20260807 19:37:35.402158 16845 ts_tablet_manager.cc:1939] T 85bbca9d7fe145ebabbcd65fec03d068 P bf677697f239490f9e29a661be949419: Deleting consensus metadata
I20260807 19:37:35.402240 16515 tablet_service.cc:1467] Processing CreateTablet for tablet 079d0cbcc78b4474b100e22c3c05b8dc (DEFAULT_TABLE table=test-table [id=1ce63dd1e0cf40b291f720ea2c994d14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:35.402638 16515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 079d0cbcc78b4474b100e22c3c05b8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:35.402951 16661 tablet_service.cc:1467] Processing CreateTablet for tablet 079d0cbcc78b4474b100e22c3c05b8dc (DEFAULT_TABLE table=test-table [id=1ce63dd1e0cf40b291f720ea2c994d14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:35.403332 16207 catalog_manager.cc:4841] TS bf677697f239490f9e29a661be949419 (127.14.16.67:33787): tablet 85bbca9d7fe145ebabbcd65fec03d068 (table test-table [id=e0a02db0b2b040c88f63bd5c1907c52f]) successfully deleted
I20260807 19:37:35.403419 16661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 079d0cbcc78b4474b100e22c3c05b8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:35.404186 16207 catalog_manager.cc:4841] TS c73aa055374b435eba0c9a1c60702c6f (127.14.16.65:39899): tablet 85bbca9d7fe145ebabbcd65fec03d068 (table test-table [id=e0a02db0b2b040c88f63bd5c1907c52f]) successfully deleted
I20260807 19:37:35.404425 16207 catalog_manager.cc:4841] TS 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403): tablet 85bbca9d7fe145ebabbcd65fec03d068 (table test-table [id=e0a02db0b2b040c88f63bd5c1907c52f]) successfully deleted
I20260807 19:37:35.404450 16368 tablet_service.cc:1467] Processing CreateTablet for tablet 079d0cbcc78b4474b100e22c3c05b8dc (DEFAULT_TABLE table=test-table [id=1ce63dd1e0cf40b291f720ea2c994d14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:35.404861 16368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 079d0cbcc78b4474b100e22c3c05b8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:35.408349 16858 tablet_bootstrap.cc:492] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419: Bootstrap starting.
I20260807 19:37:35.408737 16857 tablet_bootstrap.cc:492] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f: Bootstrap starting.
I20260807 19:37:35.410107 16858 tablet_bootstrap.cc:654] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:35.410293 16857 tablet_bootstrap.cc:654] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:35.410935 16856 tablet_bootstrap.cc:492] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f: Bootstrap starting.
I20260807 19:37:35.412667 16858 tablet_bootstrap.cc:492] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419: No bootstrap required, opened a new log
I20260807 19:37:35.412856 16858 ts_tablet_manager.cc:1397] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 19:37:35.412871 16856 tablet_bootstrap.cc:654] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:35.412909 16857 tablet_bootstrap.cc:492] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f: No bootstrap required, opened a new log
I20260807 19:37:35.413057 16857 ts_tablet_manager.cc:1397] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 19:37:35.413508 16858 raft_consensus.cc:348] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.413731 16858 raft_consensus.cc:374] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:35.413837 16858 raft_consensus.cc:729] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf677697f239490f9e29a661be949419, State: Initialized, Role: FOLLOWER
I20260807 19:37:35.413874 16857 raft_consensus.cc:348] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.414081 16858 consensus_queue.cc:260] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.414232 16857 raft_consensus.cc:374] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:35.414353 16857 raft_consensus.cc:729] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5878be99c7a94bee9b7ea0ac718e7d9f, State: Initialized, Role: FOLLOWER
I20260807 19:37:35.414642 16858 ts_tablet_manager.cc:1428] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 19:37:35.414644 16857 consensus_queue.cc:260] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.415499 16857 ts_tablet_manager.cc:1428] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 19:37:35.416944 16856 tablet_bootstrap.cc:492] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f: No bootstrap required, opened a new log
I20260807 19:37:35.417128 16856 ts_tablet_manager.cc:1397] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260807 19:37:35.417883 16856 raft_consensus.cc:348] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.418130 16856 raft_consensus.cc:374] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:35.418219 16856 raft_consensus.cc:729] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73aa055374b435eba0c9a1c60702c6f, State: Initialized, Role: FOLLOWER
I20260807 19:37:35.418512 16856 consensus_queue.cc:260] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.419662 16856 ts_tablet_manager.cc:1428] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 19:37:35.443710 16830 raft_consensus.cc:482] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:35.443964 16830 raft_consensus.cc:504] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.444618 16830 leader_election.cc:288] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403), bf677697f239490f9e29a661be949419 (127.14.16.67:33787)
I20260807 19:37:35.445080 16539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079d0cbcc78b4474b100e22c3c05b8dc" candidate_uuid: "c73aa055374b435eba0c9a1c60702c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" is_pre_election: true
I20260807 19:37:35.445241 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079d0cbcc78b4474b100e22c3c05b8dc" candidate_uuid: "c73aa055374b435eba0c9a1c60702c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf677697f239490f9e29a661be949419" is_pre_election: true
I20260807 19:37:35.445310 16539 raft_consensus.cc:2454] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c73aa055374b435eba0c9a1c60702c6f in term 0.
I20260807 19:37:35.445484 16688 raft_consensus.cc:2454] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c73aa055374b435eba0c9a1c60702c6f in term 0.
I20260807 19:37:35.445820 16318 leader_election.cc:302] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5878be99c7a94bee9b7ea0ac718e7d9f, c73aa055374b435eba0c9a1c60702c6f; no voters:
I20260807 19:37:35.446084 16830 raft_consensus.cc:2781] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:37:35.446168 16830 raft_consensus.cc:482] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:37:35.446327 16830 raft_consensus.cc:3037] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:35.448679 16830 raft_consensus.cc:504] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.449215 16830 leader_election.cc:288] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [CANDIDATE]: Term 1 election: Requested vote from peers 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403), bf677697f239490f9e29a661be949419 (127.14.16.67:33787)
I20260807 19:37:35.449652 16539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079d0cbcc78b4474b100e22c3c05b8dc" candidate_uuid: "c73aa055374b435eba0c9a1c60702c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f"
I20260807 19:37:35.449856 16539 raft_consensus.cc:3037] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:35.449828 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079d0cbcc78b4474b100e22c3c05b8dc" candidate_uuid: "c73aa055374b435eba0c9a1c60702c6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf677697f239490f9e29a661be949419"
I20260807 19:37:35.450066 16688 raft_consensus.cc:3037] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:35.452072 16539 raft_consensus.cc:2454] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c73aa055374b435eba0c9a1c60702c6f in term 1.
I20260807 19:37:35.452314 16688 raft_consensus.cc:2454] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c73aa055374b435eba0c9a1c60702c6f in term 1.
I20260807 19:37:35.452498 16318 leader_election.cc:302] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5878be99c7a94bee9b7ea0ac718e7d9f, c73aa055374b435eba0c9a1c60702c6f; no voters:
I20260807 19:37:35.452785 16830 raft_consensus.cc:2781] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:35.452960 16830 raft_consensus.cc:686] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 1 LEADER]: Becoming Leader. State: Replica: c73aa055374b435eba0c9a1c60702c6f, State: Running, Role: LEADER
I20260807 19:37:35.453250 16830 consensus_queue.cc:237] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } }
I20260807 19:37:35.455992 16220 catalog_manager.cc:5495] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f reported cstate change: term changed from 0 to 1, leader changed from <none> to c73aa055374b435eba0c9a1c60702c6f (127.14.16.65). New cstate: current_term: 1 leader_uuid: "c73aa055374b435eba0c9a1c60702c6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73aa055374b435eba0c9a1c60702c6f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 } health_report { overall_health: UNKNOWN } } }
I20260807 19:37:35.482306 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:35.477393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57238 (local address 127.14.16.65:39899)
0807 19:37:35.478018 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:35.478026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:35.478054 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:37:35.478119 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:35.478122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:35.478183 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:35.478270 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:35.478600 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:35.479411 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:35.480662 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:35.481089 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:35.481181 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:35.481554 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:35.481661 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:35.481743 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:35.481872 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 19:37:35.481932 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":377,"threads_started":1}
I20260807 19:37:35.495908 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:35.490793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:34315 (local address 127.14.16.65:39899)
0807 19:37:35.490984 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:35.490990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:35.491098 (+ 108us) server_negotiation.cc:407] Connection header received
0807 19:37:35.491309 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:35.491315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:35.491383 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:35.491477 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:35.491966 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:35.492694 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:35.494187 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:35.494607 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:35.494750 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:35.495140 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:35.495232 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:35.495350 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:35.495489 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 19:37:35.495553 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":62}
I20260807 19:37:35.495983 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:35.490423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39899 (local address 127.14.16.65:34315)
0807 19:37:35.490999 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:35.491018 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:35.491172 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:35.491489 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:35.491493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:35.491503 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:37:35.491829 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:35.491838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:35.492829 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:35.492835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:35.493882 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:35.493893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:35.494806 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:35.494814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:35.494947 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:35.494981 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:37:35.495358 (+ 377us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:37:35.495370 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:37:35.495468 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 19:37:35.495591 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":2,"thread_start_us":301,"threads_started":1}
I20260807 19:37:35.503986 16688 raft_consensus.cc:1261] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Refusing update from remote peer c73aa055374b435eba0c9a1c60702c6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:35.504136 16539 raft_consensus.cc:1261] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Refusing update from remote peer c73aa055374b435eba0c9a1c60702c6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:35.504567 16825 consensus_queue.cc:1035] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf677697f239490f9e29a661be949419" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:35.505125 16825 consensus_queue.cc:1035] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5878be99c7a94bee9b7ea0ac718e7d9f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:37:35.527944 16822 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 19:37:37.123044 16219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:37.123277 16219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:37.127243 16219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8723cbb8088b42a8ba1a9ba6dc3be206: Sending DeleteTablet for 3 replicas of tablet 079d0cbcc78b4474b100e22c3c05b8dc
I20260807 19:37:37.128047 16368 tablet_service.cc:1514] Processing DeleteTablet for tablet 079d0cbcc78b4474b100e22c3c05b8dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:37 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:57222
I20260807 19:37:37.129695 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 16297
I20260807 19:37:37.132422 16661 tablet_service.cc:1514] Processing DeleteTablet for tablet 079d0cbcc78b4474b100e22c3c05b8dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:37 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:38994
I20260807 19:37:37.132942 16515 tablet_service.cc:1514] Processing DeleteTablet for tablet 079d0cbcc78b4474b100e22c3c05b8dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:37 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:59672
I20260807 19:37:37.140928 16907 tablet_replica.cc:330] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419: stopping tablet replica
I20260807 19:37:37.141208 16907 raft_consensus.cc:2229] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:37.141518 16907 raft_consensus.cc:2258] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:37.144375 16907 ts_tablet_manager.cc:1905] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:37.146539 16908 tablet_replica.cc:330] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f: stopping tablet replica
I20260807 19:37:37.146935 16908 raft_consensus.cc:2229] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:37.147274 16908 raft_consensus.cc:2258] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:37.148332 16908 ts_tablet_manager.cc:1905] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:37.152952 16907 ts_tablet_manager.cc:1918] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:37:37.156255 16908 ts_tablet_manager.cc:1918] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:37:37.156867 16904 tablet_replica.cc:330] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f: stopping tablet replica
I20260807 19:37:37.157087 16904 raft_consensus.cc:2229] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:37:37.157138 16908 log.cc:1196] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/079d0cbcc78b4474b100e22c3c05b8dc
I20260807 19:37:37.157651 16904 raft_consensus.cc:2258] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:37.157822 16908 ts_tablet_manager.cc:1939] T 079d0cbcc78b4474b100e22c3c05b8dc P 5878be99c7a94bee9b7ea0ac718e7d9f: Deleting consensus metadata
I20260807 19:37:37.158699 16904 ts_tablet_manager.cc:1905] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:37.159046 16207 catalog_manager.cc:4841] TS 5878be99c7a94bee9b7ea0ac718e7d9f (127.14.16.66:39403): tablet 079d0cbcc78b4474b100e22c3c05b8dc (table test-table [id=1ce63dd1e0cf40b291f720ea2c994d14]) successfully deleted
I20260807 19:37:37.159391 16907 log.cc:1196] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/079d0cbcc78b4474b100e22c3c05b8dc
I20260807 19:37:37.160037 16907 ts_tablet_manager.cc:1939] T 079d0cbcc78b4474b100e22c3c05b8dc P bf677697f239490f9e29a661be949419: Deleting consensus metadata
I20260807 19:37:37.161469 16207 catalog_manager.cc:4841] TS bf677697f239490f9e29a661be949419 (127.14.16.67:33787): tablet 079d0cbcc78b4474b100e22c3c05b8dc (table test-table [id=1ce63dd1e0cf40b291f720ea2c994d14]) successfully deleted
I20260807 19:37:37.166009 16904 ts_tablet_manager.cc:1918] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:37:37.166144 16904 log.cc:1196] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/079d0cbcc78b4474b100e22c3c05b8dc
I20260807 19:37:37.166631 16904 ts_tablet_manager.cc:1939] T 079d0cbcc78b4474b100e22c3c05b8dc P c73aa055374b435eba0c9a1c60702c6f: Deleting consensus metadata
I20260807 19:37:37.167758 16207 catalog_manager.cc:4841] TS c73aa055374b435eba0c9a1c60702c6f (127.14.16.65:39899): tablet 079d0cbcc78b4474b100e22c3c05b8dc (table test-table [id=1ce63dd1e0cf40b291f720ea2c994d14]) successfully deleted
Aug 07 19:37:37 dist-test-slave-07c2 krb5kdc[16175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131455, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:37:37.176697 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:37.131179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57254 (local address 127.14.16.65:39899)
0807 19:37:37.153606 (+ 22427us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:37.153615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:37.153650 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:37:37.153729 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:37.153737 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:37.153839 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:37.156519 (+ 2680us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:37.156776 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:37.158087 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:37.159982 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:37.160745 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:37.160880 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:37.170633 (+ 9753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:37.170647 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:37.170663 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:37.170695 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:37.173922 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:37.174647 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:37.174653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:37.174656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:37.174804 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:37.175260 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:37.175263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:37.175265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:37.175680 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:37.175930 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:37.176192 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 19:37:37.176413 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22237,"thread_start_us":21989,"threads_started":1}
W20260807 19:37:37.177498 16440 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57254
W20260807 19:37:37.178108 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 16297: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:37:37.178395 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 16297
I20260807 19:37:37.208943 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 16449
Aug 07 19:37:37 dist-test-slave-07c2 krb5kdc[16175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131455, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:37:37.224153 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:37.209721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59692 (local address 127.14.16.66:39403)
0807 19:37:37.210297 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:37.210302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:37.210317 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:37:37.210375 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:37.210378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:37.210443 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:37.210536 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:37.211875 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:37.212643 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:37.213863 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:37.214278 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:37.214380 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:37.218565 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:37.218581 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:37.218597 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:37.218637 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:37.221601 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:37.222237 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:37.222244 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:37.222247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:37.222376 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:37.222729 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:37.222732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:37.222734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:37.223080 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:37.223360 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:37.223704 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 19:37:37.223884 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":332,"threads_started":1}
W20260807 19:37:37.224856 16586 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59692
W20260807 19:37:37.225613 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 16449: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:37:37.225776 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 16449
I20260807 19:37:37.251890 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 16595
Aug 07 19:37:37 dist-test-slave-07c2 krb5kdc[16175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131455, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:37:37.267171 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:37.252724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.14.16.67:33787)
0807 19:37:37.253269 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:37.253273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:37.253290 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:37:37.253364 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:37.253368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:37.253482 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:37.253589 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:37.254881 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:37.255710 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:37.256870 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:37.257271 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:37.257368 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:37.261705 (+ 4337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:37.261720 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:37.261737 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:37.261768 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:37.264778 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:37.265396 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:37.265404 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:37.265406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:37.265534 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:37.265893 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:37.265896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:37.265898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:37.266215 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:37.266436 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:37.266720 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 19:37:37.266901 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":274,"threads_started":1}
W20260807 19:37:37.267920 16730 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39016
W20260807 19:37:37.268510 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 16595: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:37:37.268673 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 16595
I20260807 19:37:37.294765 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 16188
I20260807 19:37:37.295997 16263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:37.414203 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 16188
2026-08-07T19:37:37Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8658 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:37:37 dist-test-slave-07c2 krb5kdc[16914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:37:37 dist-test-slave-07c2 krb5kdc[16914](info): set up 2 sockets
Aug 07 19:37:37 dist-test-slave-07c2 krb5kdc[16914](info): commencing operation
krb5kdc: starting...
W20260807 19:37:38.534466 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:37:38 dist-test-slave-07c2 krb5kdc[16914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:37:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:37:38.748603 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:33609
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:33609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:39.073951 16932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:39.074227 16932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:39.074321 16932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:39.081463 16932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:39.081578 16932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:39.081624 16932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:39.081678 16932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:39.081722 16932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:39.091951 16932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:33609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:33609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:39.093196 16932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:39.094444 16932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:39.106555 16940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:39.106606 16938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:39.106511 16937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:39.108784 16932 server_base.cc:1034] running on GCE node
I20260807 19:37:39.109614 16932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:39.111038 16932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:39.112283 16932 hybrid_clock.cc:648] HybridClock initialized: now 1786131459112224 us; error 71 us; skew 500 ppm
I20260807 19:37:39.112641 16932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:39 dist-test-slave-07c2 krb5kdc[16914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131459, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:39.115916 16932 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:37:39.117579 16932 webserver.cc:460] Webserver started at http://127.14.16.126:36921/ using document root <none> and password file <none>
I20260807 19:37:39.118307 16932 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:39.118458 16932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:39.118781 16932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:39.121822 16932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "3d58384f0db04621ba0c4c330ebfe207"
format_stamp: "Formatted at 2026-08-07 19:37:39 on dist-test-slave-07c2"
server_key: "fc990e11dbfac6f9cd62ca3f249af46a"
server_key_iv: "986b8aaf68a40bb96dc7ed061d4126d2"
server_key_version: "encryptionkey@0"
I20260807 19:37:39.122601 16932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "3d58384f0db04621ba0c4c330ebfe207"
format_stamp: "Formatted at 2026-08-07 19:37:39 on dist-test-slave-07c2"
server_key: "fc990e11dbfac6f9cd62ca3f249af46a"
server_key_iv: "986b8aaf68a40bb96dc7ed061d4126d2"
server_key_version: "encryptionkey@0"
I20260807 19:37:39.127856 16932 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 19:37:39.131181 16950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:39.132488 16932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:37:39.132684 16932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "3d58384f0db04621ba0c4c330ebfe207"
format_stamp: "Formatted at 2026-08-07 19:37:39 on dist-test-slave-07c2"
server_key: "fc990e11dbfac6f9cd62ca3f249af46a"
server_key_iv: "986b8aaf68a40bb96dc7ed061d4126d2"
server_key_version: "encryptionkey@0"
I20260807 19:37:39.132885 16932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:39.153288 16932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:39.157922 16932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:39.158228 16932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:39.177439 16932 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:33609
I20260807 19:37:39.177464 17004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:33609 every 8 connection(s)
I20260807 19:37:39.178958 16932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:39.182847 17005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:39.188732 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 16932
I20260807 19:37:39.188988 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:39.189638 14401 external_mini_cluster.cc:1428] Setting key d6b3243bf1d0ecd3e748e0150eb0de40
I20260807 19:37:39.194585 17005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207: Bootstrap starting.
I20260807 19:37:39.198486 17005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:39.199846 17005 log.cc:824] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:39.203214 17005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207: No bootstrap required, opened a new log
Aug 07 19:37:39 dist-test-slave-07c2 krb5kdc[16914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:39.213615 17005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d58384f0db04621ba0c4c330ebfe207" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33609 } }
I20260807 19:37:39.214082 17005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:39.214188 17005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d58384f0db04621ba0c4c330ebfe207, State: Initialized, Role: FOLLOWER
I20260807 19:37:39.214860 17005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d58384f0db04621ba0c4c330ebfe207" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33609 } }
I20260807 19:37:39.215134 17005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:39.215320 17005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:39.215538 17008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:39.191814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53840 (local address 127.14.16.126:33609)
0807 19:37:39.192729 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:39.192745 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:39.192801 (+ 56us) server_negotiation.cc:407] Connection header received
0807 19:37:39.195265 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:39.195322 (+ 57us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:39.195810 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:39.196251 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:39.197577 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:39.199650 (+ 2073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:39.201335 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:39.202382 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:39.202570 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:39.206678 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:39.206688 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:39.206707 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:39.206777 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:39.210875 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:39.211614 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:39.211623 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:39.211632 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:39.211872 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:39.212260 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:39.212265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:39.212268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:39.213023 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:39.213312 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:39.213672 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 19:37:39.214134 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":447,"threads_started":1}
I20260807 19:37:39.215535 17005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:39.218506 17005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d58384f0db04621ba0c4c330ebfe207" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33609 } }
I20260807 19:37:39.218973 17005 leader_election.cc:302] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d58384f0db04621ba0c4c330ebfe207; no voters:
I20260807 19:37:39.219563 17005 leader_election.cc:288] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:39.219712 17010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:39.220445 17010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [term 1 LEADER]: Becoming Leader. State: Replica: 3d58384f0db04621ba0c4c330ebfe207, State: Running, Role: LEADER
I20260807 19:37:39.220983 17010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d58384f0db04621ba0c4c330ebfe207" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33609 } }
I20260807 19:37:39.221637 17005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:39.223995 17011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d58384f0db04621ba0c4c330ebfe207" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d58384f0db04621ba0c4c330ebfe207" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33609 } } }
I20260807 19:37:39.224355 17011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:39.225862 17012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d58384f0db04621ba0c4c330ebfe207. Latest consensus state: current_term: 1 leader_uuid: "3d58384f0db04621ba0c4c330ebfe207" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d58384f0db04621ba0c4c330ebfe207" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33609 } } }
I20260807 19:37:39.226150 17012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:39.227551 17016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:39.233354 17016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:39.241747 17016 catalog_manager.cc:1347] Generated new cluster ID: d7b50845dc444c5297a6488a442986d8
I20260807 19:37:39.241849 17016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:39.252365 17016 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:39.253561 17016 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:39.264567 17016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d58384f0db04621ba0c4c330ebfe207: Generated new TSK 0
I20260807 19:37:39.265538 17016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:37:39.329336 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:33609
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:39.682227 17032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:39.682499 17032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:39.682598 17032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:39.689684 17032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:39.689797 17032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:39.689910 17032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:37:39.699426 17032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:33609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:39.700599 17032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:39.701856 17032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:39.715373 17037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:39.715826 17038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:39.716277 17040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:39.716890 17032 server_base.cc:1034] running on GCE node
I20260807 19:37:39.717490 17032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:39.718492 17032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:39.719722 17032 hybrid_clock.cc:648] HybridClock initialized: now 1786131459719674 us; error 62 us; skew 500 ppm
I20260807 19:37:39.720042 17032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:39 dist-test-slave-07c2 krb5kdc[16914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131459, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:39.723487 17032 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:37:39.725005 17032 webserver.cc:460] Webserver started at http://127.14.16.65:44979/ using document root <none> and password file <none>
I20260807 19:37:39.725792 17032 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:39.725934 17032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:39.726229 17032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:39.729257 17032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "462b797fb59d4d14aa9087f14cd89c4c"
format_stamp: "Formatted at 2026-08-07 19:37:39 on dist-test-slave-07c2"
server_key: "c848a958b038e0cf30075cd25ed65317"
server_key_iv: "29635052aeb6e542ce216ddb265f2334"
server_key_version: "encryptionkey@0"
I20260807 19:37:39.730050 17032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "462b797fb59d4d14aa9087f14cd89c4c"
format_stamp: "Formatted at 2026-08-07 19:37:39 on dist-test-slave-07c2"
server_key: "c848a958b038e0cf30075cd25ed65317"
server_key_iv: "29635052aeb6e542ce216ddb265f2334"
server_key_version: "encryptionkey@0"
I20260807 19:37:39.735520 17032 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 19:37:39.739063 17047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:39.740396 17032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:37:39.740525 17032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "462b797fb59d4d14aa9087f14cd89c4c"
format_stamp: "Formatted at 2026-08-07 19:37:39 on dist-test-slave-07c2"
server_key: "c848a958b038e0cf30075cd25ed65317"
server_key_iv: "29635052aeb6e542ce216ddb265f2334"
server_key_version: "encryptionkey@0"
I20260807 19:37:39.740705 17032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:39.767221 17032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:39.771781 17032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:39.772027 17032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:39.772964 17032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:39.774272 17032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:39.774344 17032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:39.774430 17032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:39.774474 17032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:39.808560 17032 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:36883
I20260807 19:37:39.808610 17167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:36883 every 8 connection(s)
I20260807 19:37:39.809847 17032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:37:39.814433 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 17032
I20260807 19:37:39.814666 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:37:39.815348 14401 external_mini_cluster.cc:1428] Setting key e26283729a12cae51a2d76f874fc793d
Aug 07 19:37:39 dist-test-slave-07c2 krb5kdc[16914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131459, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:39.830092 17170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:39.812408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:47563 (local address 127.14.16.126:33609)
0807 19:37:39.812889 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:39.812893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:39.813665 (+ 772us) server_negotiation.cc:407] Connection header received
0807 19:37:39.815688 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:39.815694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:39.815789 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:39.815908 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:39.817626 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:39.818359 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:39.819929 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:39.820318 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:39.820457 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:39.823997 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:39.824005 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:39.824013 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:39.824057 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:39.826762 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:39.827742 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:39.827746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:39.827748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:39.827864 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:39.828335 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:39.828338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:39.828340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:39.828606 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:39.828761 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:39.829583 (+ 822us) server_negotiation.cc:299] Negotiation successful
0807 19:37:39.829797 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":255,"threads_started":1}
I20260807 19:37:39.831610 17172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:39.812823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:33609 (local address 127.14.16.65:47563)
0807 19:37:39.813501 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:39.813545 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:39.815400 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:39.816139 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:39.816146 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:39.816855 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:39.817459 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:39.817477 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:39.818524 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:39.818528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:39.819775 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:39.819791 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:39.820547 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:39.820557 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:39.820788 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:39.821455 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:39.821477 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:39.823779 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:39.826907 (+ 3128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:39.826918 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:39.826935 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:39.827617 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:39.827971 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:39.827978 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:39.827984 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:39.828225 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:39.828732 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:39.828742 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:39.829178 (+ 436us) client_negotiation.cc:769] Sending connection context
0807 19:37:39.829482 (+ 304us) client_negotiation.cc:240] Negotiation successful
0807 19:37:39.830098 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":262,"threads_started":1}
I20260807 19:37:39.833210 17168 heartbeater.cc:347] Connected to a master server at 127.14.16.126:33609
I20260807 19:37:39.833552 17168 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:39.834450 17168 heartbeater.cc:511] Master 127.14.16.126:33609 requested a full tablet report, sending...
I20260807 19:37:39.836964 16969 ts_manager.cc:194] Registered new tserver with Master: 462b797fb59d4d14aa9087f14cd89c4c (127.14.16.65:36883)
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
I20260807 19:37:39.838917 16969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:47563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:37:39.864022 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:33609
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:40.187492 17175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:40.187836 17175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:40.187984 17175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:40.199580 17175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:40.199709 17175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:40.199887 17175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:37:40.209949 17175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:33609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:40.211611 17175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:40.212963 17175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:40.225741 17180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:40.225759 17183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:40.225688 17181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:40.226481 17175 server_base.cc:1034] running on GCE node
I20260807 19:37:40.227061 17175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:40.228063 17175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:40.229295 17175 hybrid_clock.cc:648] HybridClock initialized: now 1786131460229232 us; error 40 us; skew 500 ppm
I20260807 19:37:40.229635 17175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:40 dist-test-slave-07c2 krb5kdc[16914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131460, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:40.232999 17175 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:37:40.234633 17175 webserver.cc:460] Webserver started at http://127.14.16.66:34637/ using document root <none> and password file <none>
I20260807 19:37:40.235330 17175 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:40.235488 17175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:40.235805 17175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:40.238960 17175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "f51745c15f604337a647439a69abfbf8"
format_stamp: "Formatted at 2026-08-07 19:37:40 on dist-test-slave-07c2"
server_key: "27884d7c6e7d40afb607b75ad4967dda"
server_key_iv: "089b8e32b0a8b3618e5fabb212a003ed"
server_key_version: "encryptionkey@0"
I20260807 19:37:40.239768 17175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "f51745c15f604337a647439a69abfbf8"
format_stamp: "Formatted at 2026-08-07 19:37:40 on dist-test-slave-07c2"
server_key: "27884d7c6e7d40afb607b75ad4967dda"
server_key_iv: "089b8e32b0a8b3618e5fabb212a003ed"
server_key_version: "encryptionkey@0"
I20260807 19:37:40.244972 17175 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 19:37:40.248245 17192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:40.249516 17175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:37:40.249684 17175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "f51745c15f604337a647439a69abfbf8"
format_stamp: "Formatted at 2026-08-07 19:37:40 on dist-test-slave-07c2"
server_key: "27884d7c6e7d40afb607b75ad4967dda"
server_key_iv: "089b8e32b0a8b3618e5fabb212a003ed"
server_key_version: "encryptionkey@0"
I20260807 19:37:40.249891 17175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:40.260560 17175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:40.264503 17175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:40.264798 17175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:40.265681 17175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:40.266945 17175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:40.267050 17175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:40.267201 17175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:40.267275 17175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:40.301448 17175 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:40859
I20260807 19:37:40.301507 17307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:40859 every 8 connection(s)
I20260807 19:37:40.302814 17175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:37:40.312601 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 17175
I20260807 19:37:40.312803 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:37:40.313385 14401 external_mini_cluster.cc:1428] Setting key 0da2675644576a859c2d9d70febc57f0
Aug 07 19:37:40 dist-test-slave-07c2 krb5kdc[16914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131460, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:40.323598 17170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:40.305351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:54623 (local address 127.14.16.126:33609)
0807 19:37:40.305560 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:40.305566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:40.306719 (+ 1153us) server_negotiation.cc:407] Connection header received
0807 19:37:40.308670 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:40.308674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:40.308745 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:40.308835 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:40.310239 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:40.310922 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:40.312460 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:40.313141 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:40.313283 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:40.316689 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:40.316698 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:40.316706 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:40.316773 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:40.320434 (+ 3661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:40.321126 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:40.321134 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:40.321140 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:40.321352 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:40.321776 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:40.321783 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:40.321789 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:40.322166 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:40.322324 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:40.323035 (+ 711us) server_negotiation.cc:299] Negotiation successful
0807 19:37:40.323342 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:37:40.324884 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:40.305869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:33609 (local address 127.14.16.66:54623)
0807 19:37:40.306562 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:40.306604 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:40.308418 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:40.309023 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:40.309030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:40.309563 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:40.310087 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:40.310103 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:40.311053 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:40.311057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:40.312305 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:40.312317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:40.313349 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:40.313354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:40.313581 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:40.314259 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:40.314280 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:40.316458 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:40.320565 (+ 4107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:40.320572 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:40.320583 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:40.320992 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:40.321463 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:40.321466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:40.321469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:40.321651 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:40.322296 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:40.322302 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:40.322656 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 19:37:40.322876 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 19:37:40.323434 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":275,"threads_started":1}
I20260807 19:37:40.326489 17308 heartbeater.cc:347] Connected to a master server at 127.14.16.126:33609
I20260807 19:37:40.326815 17308 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:40.327725 17308 heartbeater.cc:511] Master 127.14.16.126:33609 requested a full tablet report, sending...
I20260807 19:37:40.329348 16969 ts_manager.cc:194] Registered new tserver with Master: f51745c15f604337a647439a69abfbf8 (127.14.16.66:40859)
I20260807 19:37:40.330250 16969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:54623
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:37:40.362921 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:33609
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:40.686395 17314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:40.686717 17314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:40.686889 17314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:40.694049 17314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:40.694206 17314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:40.694380 17314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:37:40.704388 17314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:33609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:40.705632 17314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:40.706952 17314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:40.719552 17320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:40.719628 17322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:40.719669 17319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:40.720355 17314 server_base.cc:1034] running on GCE node
I20260807 19:37:40.720980 17314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:40.721978 17314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:40.723272 17314 hybrid_clock.cc:648] HybridClock initialized: now 1786131460723212 us; error 41 us; skew 500 ppm
I20260807 19:37:40.723634 17314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:40 dist-test-slave-07c2 krb5kdc[16914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131460, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:40.726876 17314 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:37:40.728355 17314 webserver.cc:460] Webserver started at http://127.14.16.67:38745/ using document root <none> and password file <none>
I20260807 19:37:40.729112 17314 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:40.729260 17314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:40.729642 17314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:40.732722 17314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "8f67530069db41b08b7fec23b348b94f"
format_stamp: "Formatted at 2026-08-07 19:37:40 on dist-test-slave-07c2"
server_key: "750d013fde27c8c6fe095b5bf422e2b6"
server_key_iv: "13dba4594b7615f655fa8d26bb5ba151"
server_key_version: "encryptionkey@0"
I20260807 19:37:40.733523 17314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "8f67530069db41b08b7fec23b348b94f"
format_stamp: "Formatted at 2026-08-07 19:37:40 on dist-test-slave-07c2"
server_key: "750d013fde27c8c6fe095b5bf422e2b6"
server_key_iv: "13dba4594b7615f655fa8d26bb5ba151"
server_key_version: "encryptionkey@0"
I20260807 19:37:40.738831 17314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:37:40.742405 17329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:40.743805 17314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:37:40.743978 17314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "8f67530069db41b08b7fec23b348b94f"
format_stamp: "Formatted at 2026-08-07 19:37:40 on dist-test-slave-07c2"
server_key: "750d013fde27c8c6fe095b5bf422e2b6"
server_key_iv: "13dba4594b7615f655fa8d26bb5ba151"
server_key_version: "encryptionkey@0"
I20260807 19:37:40.744177 17314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:40.759371 17314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:40.763273 17314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:40.763612 17314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:40.764520 17314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:40.765818 17314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:40.765923 17314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:40.766072 17314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:40.766151 17314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:40.799332 17314 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:43609
I20260807 19:37:40.799391 17444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:43609 every 8 connection(s)
I20260807 19:37:40.800614 17314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:37:40.801743 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 17314
I20260807 19:37:40.801963 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:37:40.802531 14401 external_mini_cluster.cc:1428] Setting key 5f272b15f40de2ecd4237171de08c89c
Aug 07 19:37:40 dist-test-slave-07c2 krb5kdc[16914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131460, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:40.822149 17170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:40.803743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:54697 (local address 127.14.16.126:33609)
0807 19:37:40.803935 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:40.803942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:40.805000 (+ 1058us) server_negotiation.cc:407] Connection header received
0807 19:37:40.807016 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:40.807021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:40.807113 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:40.807222 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:40.809217 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:40.809945 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:40.812203 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:40.812607 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:40.812703 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:40.816336 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:40.816344 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:40.816351 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:40.816398 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:40.819404 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:40.820052 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:40.820055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:40.820057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:40.820198 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:40.820635 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:40.820643 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:40.820647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:40.820982 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:40.821094 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:40.821755 (+ 661us) server_negotiation.cc:299] Negotiation successful
0807 19:37:40.821911 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:37:40.823117 17447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:40.804024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:33609 (local address 127.14.16.67:54697)
0807 19:37:40.804832 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:40.804891 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:40.806766 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:40.807471 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:40.807479 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:40.808229 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:40.809017 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:40.809048 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:40.810109 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:40.810116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:40.812067 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:40.812083 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:40.812850 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:40.812858 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:40.813127 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:40.813830 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:40.813860 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:40.816108 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:40.819528 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:40.819535 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:40.819545 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:40.819937 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:40.820308 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:40.820312 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:40.820314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:40.820489 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:40.821090 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:40.821095 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:40.821425 (+ 330us) client_negotiation.cc:769] Sending connection context
0807 19:37:40.821677 (+ 252us) client_negotiation.cc:240] Negotiation successful
0807 19:37:40.822024 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":317,"threads_started":1}
I20260807 19:37:40.824401 17445 heartbeater.cc:347] Connected to a master server at 127.14.16.126:33609
I20260807 19:37:40.824641 17445 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:40.825253 17445 heartbeater.cc:511] Master 127.14.16.126:33609 requested a full tablet report, sending...
I20260807 19:37:40.826620 16969 ts_manager.cc:194] Registered new tserver with Master: 8f67530069db41b08b7fec23b348b94f (127.14.16.67:43609)
I20260807 19:37:40.827442 16969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:54697
I20260807 19:37:40.830663 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:37:40.841818 17168 heartbeater.cc:503] Master 127.14.16.126:33609 was elected leader, sending a full tablet report...
W20260807 19:37:40.866649 17170 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:40.864171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53854 (local address 127.14.16.126:33609)
0807 19:37:40.864449 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:40.864456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:40.864759 (+ 303us) server_negotiation.cc:407] Connection header received
0807 19:37:40.864912 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:40.864916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:40.864983 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:40.865072 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:40.866359 (+ 1287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53854: BlockingRecv error: recv got EOF from 127.0.0.1:53854 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 19:37:40.869675 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17032
W20260807 19:37:40.872400 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17032: Not authorized: Client connection negotiation failed: client connection to 127.14.16.65:36883: server requires authentication, but client does not have Kerberos credentials available
I20260807 19:37:40.872589 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 17032
I20260807 19:37:40.890588 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17175
W20260807 19:37:40.893261 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17175: Not authorized: Client connection negotiation failed: client connection to 127.14.16.66:40859: server requires authentication, but client does not have Kerberos credentials available
I20260807 19:37:40.893435 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 17175
I20260807 19:37:40.911559 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17314
W20260807 19:37:40.915995 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17314: Not authorized: Client connection negotiation failed: client connection to 127.14.16.67:43609: server requires authentication, but client does not have Kerberos credentials available
I20260807 19:37:40.916165 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 17314
I20260807 19:37:40.934536 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 16932
I20260807 19:37:40.935724 17000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:41.041045 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 16932
2026-08-07T19:37:41Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3624 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:37:41 dist-test-slave-07c2 krb5kdc[17463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:37:41 dist-test-slave-07c2 krb5kdc[17463](info): set up 2 sockets
Aug 07 19:37:41 dist-test-slave-07c2 krb5kdc[17463](info): commencing operation
krb5kdc: starting...
W20260807 19:37:42.151064 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:37:42 dist-test-slave-07c2 krb5kdc[17463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:37:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:37:42.362437 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:33501
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:46645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:33501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:42.692157 17476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:42.692456 17476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:42.692548 17476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:42.699700 17476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:42.699807 17476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:42.699858 17476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:42.699916 17476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:42.699957 17476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:42.709683 17476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:46645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:33501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:33501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:42.710851 17476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:42.712085 17476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:42.724104 17482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:42.724186 17483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:42.724220 17485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:42.725013 17476 server_base.cc:1034] running on GCE node
I20260807 19:37:42.725701 17476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:42.726912 17476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:42.728106 17476 hybrid_clock.cc:648] HybridClock initialized: now 1786131462728049 us; error 45 us; skew 500 ppm
I20260807 19:37:42.728449 17476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:42 dist-test-slave-07c2 krb5kdc[17463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131462, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:42.739195 17476 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:37:42.740674 17476 webserver.cc:460] Webserver started at http://127.14.16.126:39947/ using document root <none> and password file <none>
I20260807 19:37:42.741344 17476 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:42.741495 17476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:42.741807 17476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:42.747223 17476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "1cbeaea3067043809fc5014fc81b69b2"
format_stamp: "Formatted at 2026-08-07 19:37:42 on dist-test-slave-07c2"
server_key: "bb9727a013241426e17353dd315454f1"
server_key_iv: "3c46decdfa0860603021fe66dec01614"
server_key_version: "encryptionkey@0"
I20260807 19:37:42.748112 17476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "1cbeaea3067043809fc5014fc81b69b2"
format_stamp: "Formatted at 2026-08-07 19:37:42 on dist-test-slave-07c2"
server_key: "bb9727a013241426e17353dd315454f1"
server_key_iv: "3c46decdfa0860603021fe66dec01614"
server_key_version: "encryptionkey@0"
I20260807 19:37:42.753326 17476 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 19:37:42.756798 17492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:42.758054 17476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:37:42.758246 17476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "1cbeaea3067043809fc5014fc81b69b2"
format_stamp: "Formatted at 2026-08-07 19:37:42 on dist-test-slave-07c2"
server_key: "bb9727a013241426e17353dd315454f1"
server_key_iv: "3c46decdfa0860603021fe66dec01614"
server_key_version: "encryptionkey@0"
I20260807 19:37:42.758460 17476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:42.801141 17476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:42.805270 17476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:42.805564 17476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:42.823562 17476 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:33501
I20260807 19:37:42.823594 17545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:33501 every 8 connection(s)
I20260807 19:37:42.824920 17476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:42.828706 17546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:42.832049 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 17476
I20260807 19:37:42.832288 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:42.832930 14401 external_mini_cluster.cc:1428] Setting key 91bd0d8a390e3e0ccb5979f71b7e7edb
I20260807 19:37:42.841199 17546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2: Bootstrap starting.
I20260807 19:37:42.844939 17546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:42.846247 17546 log.cc:824] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:37:42 dist-test-slave-07c2 krb5kdc[17463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:42.849205 17546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2: No bootstrap required, opened a new log
I20260807 19:37:42.856822 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:42.834655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.14.16.126:33501)
0807 19:37:42.835462 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:42.835473 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:42.835510 (+ 37us) server_negotiation.cc:407] Connection header received
0807 19:37:42.837088 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:42.837108 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:42.837469 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:42.837946 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:42.839889 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:42.841153 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:42.842949 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:42.843655 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:42.843776 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:42.848900 (+ 5124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:42.848909 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:42.848923 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:42.848958 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:42.852406 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:42.853198 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:42.853211 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:42.853217 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:42.853415 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:42.853848 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:42.853864 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:42.853870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:42.854432 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:42.854794 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:42.855321 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 19:37:42.855803 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":393,"threads_started":1}
I20260807 19:37:42.856920 17546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbeaea3067043809fc5014fc81b69b2" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33501 } }
I20260807 19:37:42.857360 17546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:42.857484 17546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cbeaea3067043809fc5014fc81b69b2, State: Initialized, Role: FOLLOWER
I20260807 19:37:42.858101 17546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbeaea3067043809fc5014fc81b69b2" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33501 } }
I20260807 19:37:42.858361 17546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:42.858518 17546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:42.858707 17546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:42.861076 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:37:42.861349 17546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbeaea3067043809fc5014fc81b69b2" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33501 } }
I20260807 19:37:42.861871 17546 leader_election.cc:302] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cbeaea3067043809fc5014fc81b69b2; no voters:
I20260807 19:37:42.862591 17546 leader_election.cc:288] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:42.862694 17551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:42.865135 17546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:42.869876 17551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [term 1 LEADER]: Becoming Leader. State: Replica: 1cbeaea3067043809fc5014fc81b69b2, State: Running, Role: LEADER
I20260807 19:37:42.870568 17551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbeaea3067043809fc5014fc81b69b2" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33501 } }
I20260807 19:37:42.873860 17552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cbeaea3067043809fc5014fc81b69b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbeaea3067043809fc5014fc81b69b2" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33501 } } }
I20260807 19:37:42.874305 17552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:42.874907 17553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cbeaea3067043809fc5014fc81b69b2. Latest consensus state: current_term: 1 leader_uuid: "1cbeaea3067043809fc5014fc81b69b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbeaea3067043809fc5014fc81b69b2" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33501 } } }
I20260807 19:37:42.875227 17553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:42.883730 17562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:42.889386 17562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:42.898522 17562 catalog_manager.cc:1347] Generated new cluster ID: e8103a31a6124f5e98c4f96c03e7e7e8
I20260807 19:37:42.898677 17562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:42.916653 17562 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:42.918448 17562 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:42.922922 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:42.913614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41662 (local address 127.14.16.126:33501)
0807 19:37:42.913803 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:42.913811 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:42.914370 (+ 559us) server_negotiation.cc:407] Connection header received
0807 19:37:42.914563 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:42.914569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:42.914653 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:42.914774 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:42.916142 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:42.917231 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:42.919164 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:42.919858 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:42.919992 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:42.921284 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:42.921291 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:42.921296 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:42.922090 (+ 794us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:42.922164 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:42.922199 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:42.922306 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:42.922479 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 19:37:42.922597 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 19:37:42.926115 17562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cbeaea3067043809fc5014fc81b69b2: Generated new TSK 0
I20260807 19:37:42.926872 17562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:37:42.935578 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17476
I20260807 19:37:42.936861 17541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:43.029695 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 17476
2026-08-07T19:37:43Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1984 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:37:43 dist-test-slave-07c2 krb5kdc[17582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:37:43 dist-test-slave-07c2 krb5kdc[17582](info): set up 2 sockets
Aug 07 19:37:43 dist-test-slave-07c2 krb5kdc[17582](info): commencing operation
krb5kdc: starting...
W20260807 19:37:44.150378 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:37:44 dist-test-slave-07c2 krb5kdc[17582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:37:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:37:44.372529 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:36847
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:36847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:44.701133 17595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:44.701443 17595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:44.701647 17595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:44.708905 17595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:44.709014 17595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:44.709131 17595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:44.709234 17595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:44.709328 17595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:44.719129 17595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:36847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:36847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:44.720487 17595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:44.721829 17595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:44.733727 17600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:44.733661 17601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:44.733700 17603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:44.734840 17595 server_base.cc:1034] running on GCE node
I20260807 19:37:44.735689 17595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:44.737088 17595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:44.738301 17595 hybrid_clock.cc:648] HybridClock initialized: now 1786131464738268 us; error 40 us; skew 500 ppm
I20260807 19:37:44.738673 17595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:44 dist-test-slave-07c2 krb5kdc[17582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131464, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:44.741783 17595 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:37:44.743453 17595 webserver.cc:460] Webserver started at http://127.14.16.126:33137/ using document root <none> and password file <none>
I20260807 19:37:44.744130 17595 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:44.744278 17595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:44.744603 17595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:44.747594 17595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "0a93a16f014d4fe2ba63e00035ca3be2"
format_stamp: "Formatted at 2026-08-07 19:37:44 on dist-test-slave-07c2"
server_key: "402c33a3d59e328658aa1ba3296955f2"
server_key_iv: "813085d97120d5a07d37b61936db5b0d"
server_key_version: "encryptionkey@0"
I20260807 19:37:44.748359 17595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "0a93a16f014d4fe2ba63e00035ca3be2"
format_stamp: "Formatted at 2026-08-07 19:37:44 on dist-test-slave-07c2"
server_key: "402c33a3d59e328658aa1ba3296955f2"
server_key_iv: "813085d97120d5a07d37b61936db5b0d"
server_key_version: "encryptionkey@0"
I20260807 19:37:44.753450 17595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:37:44.756888 17610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:44.758246 17595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:37:44.758433 17595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "0a93a16f014d4fe2ba63e00035ca3be2"
format_stamp: "Formatted at 2026-08-07 19:37:44 on dist-test-slave-07c2"
server_key: "402c33a3d59e328658aa1ba3296955f2"
server_key_iv: "813085d97120d5a07d37b61936db5b0d"
server_key_version: "encryptionkey@0"
I20260807 19:37:44.758639 17595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:44.773190 17595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:44.777302 17595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:44.777608 17595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:44.796650 17595 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:36847
I20260807 19:37:44.796690 17664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:36847 every 8 connection(s)
I20260807 19:37:44.798101 17595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:44.801909 17665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:44.803705 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 17595
I20260807 19:37:44.803926 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:44.804589 14401 external_mini_cluster.cc:1428] Setting key 6a061989ffb418ac7280318903437fd8
I20260807 19:37:44.814107 17665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2: Bootstrap starting.
I20260807 19:37:44.818006 17665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:44.819278 17665 log.cc:824] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:37:44 dist-test-slave-07c2 krb5kdc[17582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:44.823036 17665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2: No bootstrap required, opened a new log
I20260807 19:37:44.829609 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:44.806562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34958 (local address 127.14.16.126:36847)
0807 19:37:44.807432 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:44.807443 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:44.807490 (+ 47us) server_negotiation.cc:407] Connection header received
0807 19:37:44.809163 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:44.809184 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:44.809586 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:44.809890 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:44.811744 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:44.813065 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:44.814782 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:44.815504 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:44.815676 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:44.820902 (+ 5226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:44.820915 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:44.820934 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:44.820980 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:44.824981 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:44.825663 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:44.825672 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:44.825685 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:44.825910 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:44.826249 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:44.826257 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:44.826262 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:44.826996 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:44.827348 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:44.827705 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 19:37:44.828156 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":368,"threads_started":1}
I20260807 19:37:44.833029 17665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a93a16f014d4fe2ba63e00035ca3be2" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36847 } }
I20260807 19:37:44.833382 17665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:44.833458 17665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a93a16f014d4fe2ba63e00035ca3be2, State: Initialized, Role: FOLLOWER
I20260807 19:37:44.833926 17665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a93a16f014d4fe2ba63e00035ca3be2" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36847 } }
I20260807 19:37:44.834120 17665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:44.834230 17665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:44.834348 17665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:44.836624 17665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a93a16f014d4fe2ba63e00035ca3be2" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36847 } }
I20260807 19:37:44.837064 17665 leader_election.cc:302] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a93a16f014d4fe2ba63e00035ca3be2; no voters:
I20260807 19:37:44.837628 17665 leader_election.cc:288] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:44.837805 17670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:44.838609 17670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [term 1 LEADER]: Becoming Leader. State: Replica: 0a93a16f014d4fe2ba63e00035ca3be2, State: Running, Role: LEADER
I20260807 19:37:44.839190 17670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a93a16f014d4fe2ba63e00035ca3be2" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36847 } }
I20260807 19:37:44.839957 17665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:44.842067 17672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a93a16f014d4fe2ba63e00035ca3be2. Latest consensus state: current_term: 1 leader_uuid: "0a93a16f014d4fe2ba63e00035ca3be2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a93a16f014d4fe2ba63e00035ca3be2" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36847 } } }
I20260807 19:37:44.842286 17671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a93a16f014d4fe2ba63e00035ca3be2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a93a16f014d4fe2ba63e00035ca3be2" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36847 } } }
I20260807 19:37:44.842633 17672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:44.842633 17671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:44.845898 17677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:44.850775 17677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:44.859045 17677 catalog_manager.cc:1347] Generated new cluster ID: feeb57eb3ba64db8bbf9407431fb128e
I20260807 19:37:44.859164 17677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:44.866714 17677 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:44.868573 17677 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:44.878310 17677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a93a16f014d4fe2ba63e00035ca3be2: Generated new TSK 0
I20260807 19:37:44.879323 17677 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:37:44.938294 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:36847
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:45.273063 17691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:45.273409 17691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:45.273595 17691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:45.280889 17691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:45.281075 17691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:45.281265 17691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:37:45.291214 17691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:36847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:45.292636 17691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:45.294001 17691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:45.307989 17697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:45.308036 17699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:45.308075 17696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:45.308975 17691 server_base.cc:1034] running on GCE node
I20260807 19:37:45.309520 17691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:45.310438 17691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:45.311676 17691 hybrid_clock.cc:648] HybridClock initialized: now 1786131465311635 us; error 33 us; skew 500 ppm
I20260807 19:37:45.312022 17691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:45 dist-test-slave-07c2 krb5kdc[17582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131465, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:45.315388 17691 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:37:45.316758 17691 webserver.cc:460] Webserver started at http://127.14.16.65:45873/ using document root <none> and password file <none>
I20260807 19:37:45.317410 17691 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:45.317512 17691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:45.317775 17691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:45.320876 17691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "2f77a7c96f9846adafe3d56af653461b"
format_stamp: "Formatted at 2026-08-07 19:37:45 on dist-test-slave-07c2"
server_key: "1b21b19f5d1cbfd1ed3edd3500b66810"
server_key_iv: "15c4cdd976c621be22c2a1e678a942d8"
server_key_version: "encryptionkey@0"
I20260807 19:37:45.321826 17691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "2f77a7c96f9846adafe3d56af653461b"
format_stamp: "Formatted at 2026-08-07 19:37:45 on dist-test-slave-07c2"
server_key: "1b21b19f5d1cbfd1ed3edd3500b66810"
server_key_iv: "15c4cdd976c621be22c2a1e678a942d8"
server_key_version: "encryptionkey@0"
I20260807 19:37:45.327425 17691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:37:45.331568 17706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:45.333155 17691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:37:45.333365 17691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "2f77a7c96f9846adafe3d56af653461b"
format_stamp: "Formatted at 2026-08-07 19:37:45 on dist-test-slave-07c2"
server_key: "1b21b19f5d1cbfd1ed3edd3500b66810"
server_key_iv: "15c4cdd976c621be22c2a1e678a942d8"
server_key_version: "encryptionkey@0"
I20260807 19:37:45.333611 17691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:45.351506 17691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:45.355923 17691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:45.356238 17691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:45.357203 17691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:45.358595 17691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:45.358700 17691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:45.358848 17691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:45.358922 17691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:45.393121 17691 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:37493
I20260807 19:37:45.393168 17822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:37493 every 8 connection(s)
I20260807 19:37:45.394415 17691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:37:45.400924 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 17691
I20260807 19:37:45.401152 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:37:45.402014 14401 external_mini_cluster.cc:1428] Setting key 310b9bb5773695fbc714f71f2a9c423a
Aug 07 19:37:45 dist-test-slave-07c2 krb5kdc[17582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131465, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:45.417102 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:45.397233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:35019 (local address 127.14.16.126:36847)
0807 19:37:45.397707 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:45.397715 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:45.398638 (+ 923us) server_negotiation.cc:407] Connection header received
0807 19:37:45.400500 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:45.400506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:45.400602 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:45.400737 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:45.402807 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:45.403752 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:45.406136 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:45.406537 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:45.406639 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:45.410951 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:45.410956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:45.410959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:45.410992 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:45.413706 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:45.414680 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:45.414686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:45.414690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:45.414850 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:45.415331 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:45.415336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:45.415340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:45.415615 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:45.415780 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:45.416478 (+ 698us) server_negotiation.cc:299] Negotiation successful
0807 19:37:45.416724 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":224,"threads_started":1}
I20260807 19:37:45.418098 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:45.397770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:36847 (local address 127.14.16.65:35019)
0807 19:37:45.398484 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:45.398521 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:45.400178 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:45.401017 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:45.401031 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:45.401801 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:45.402603 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:45.402630 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:45.403905 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:45.403910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:45.405965 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:45.405985 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:45.406792 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:45.406802 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:45.407137 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:45.407920 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:45.407961 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:45.410708 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:45.413857 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:45.413866 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:45.413882 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:45.414528 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:45.414973 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:45.414978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:45.414981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:45.415186 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:45.415750 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:45.415758 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:45.416102 (+ 344us) client_negotiation.cc:769] Sending connection context
0807 19:37:45.416382 (+ 280us) client_negotiation.cc:240] Negotiation successful
0807 19:37:45.416927 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":276,"threads_started":1}
I20260807 19:37:45.419761 17823 heartbeater.cc:347] Connected to a master server at 127.14.16.126:36847
I20260807 19:37:45.420114 17823 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:45.421123 17823 heartbeater.cc:511] Master 127.14.16.126:36847 requested a full tablet report, sending...
I20260807 19:37:45.423768 17627 ts_manager.cc:194] Registered new tserver with Master: 2f77a7c96f9846adafe3d56af653461b (127.14.16.65:37493)
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
I20260807 19:37:45.425565 17627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:35019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:37:45.452577 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:36847
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:45.787153 17830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:45.787477 17830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:45.787662 17830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:45.794778 17830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:45.794945 17830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:45.795142 17830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:37:45.804953 17830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:36847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:45.806134 17830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:45.807420 17830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:45.820641 17835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:45.820688 17838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:45.820713 17836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:45.821460 17830 server_base.cc:1034] running on GCE node
I20260807 19:37:45.822098 17830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:45.823069 17830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:45.824301 17830 hybrid_clock.cc:648] HybridClock initialized: now 1786131465824244 us; error 72 us; skew 500 ppm
I20260807 19:37:45.824612 17830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:45 dist-test-slave-07c2 krb5kdc[17582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131465, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:45.827705 17830 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:37:45.829173 17830 webserver.cc:460] Webserver started at http://127.14.16.66:39967/ using document root <none> and password file <none>
I20260807 19:37:45.829833 17830 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:45.829984 17830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:45.830299 17830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:45.833294 17830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "21108e6dec7f4e9d9a4d21ee710a549b"
format_stamp: "Formatted at 2026-08-07 19:37:45 on dist-test-slave-07c2"
server_key: "28209cbaf497b6cb069ec65b7e77da7c"
server_key_iv: "b125e96953527c031474fe20928de181"
server_key_version: "encryptionkey@0"
I20260807 19:37:45.834075 17830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "21108e6dec7f4e9d9a4d21ee710a549b"
format_stamp: "Formatted at 2026-08-07 19:37:45 on dist-test-slave-07c2"
server_key: "28209cbaf497b6cb069ec65b7e77da7c"
server_key_iv: "b125e96953527c031474fe20928de181"
server_key_version: "encryptionkey@0"
I20260807 19:37:45.839368 17830 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:37:45.842622 17845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:45.843997 17830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 19:37:45.844169 17830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "21108e6dec7f4e9d9a4d21ee710a549b"
format_stamp: "Formatted at 2026-08-07 19:37:45 on dist-test-slave-07c2"
server_key: "28209cbaf497b6cb069ec65b7e77da7c"
server_key_iv: "b125e96953527c031474fe20928de181"
server_key_version: "encryptionkey@0"
I20260807 19:37:45.844419 17830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:45.860303 17830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:45.864398 17830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:45.864711 17830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:45.865696 17830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:45.867069 17830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:45.867177 17830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:45.867383 17830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:45.867462 17830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 19:37:45.902479 17830 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:39777
I20260807 19:37:45.902525 17959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:39777 every 8 connection(s)
I20260807 19:37:45.904078 17830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:37:45.904491 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 17830
I20260807 19:37:45.904698 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:37:45.905421 14401 external_mini_cluster.cc:1428] Setting key 020ab690debd9ce12cb4ec71545df056
Aug 07 19:37:45 dist-test-slave-07c2 krb5kdc[17582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131465, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:45.925944 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:45.906844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:54451 (local address 127.14.16.126:36847)
0807 19:37:45.907049 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:45.907058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:45.908580 (+ 1522us) server_negotiation.cc:407] Connection header received
0807 19:37:45.910358 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:45.910367 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:45.910459 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:45.910571 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:45.912049 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:45.913190 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:45.914687 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:45.915371 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:45.915470 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:45.919598 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:45.919602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:45.919606 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:45.919638 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:45.922094 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:45.923141 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:45.923150 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:45.923155 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:45.923456 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:45.923933 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:45.923940 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:45.923946 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:45.924337 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:45.924512 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:45.925176 (+ 664us) server_negotiation.cc:299] Negotiation successful
0807 19:37:45.925464 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:37:45.926429 17962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:45.907481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:36847 (local address 127.14.16.66:54451)
0807 19:37:45.908393 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:45.908439 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:45.910089 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:45.910772 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:45.910780 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:45.911259 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:45.911874 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:45.911891 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:45.913350 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:45.913354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:45.914542 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:45.914553 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:45.915603 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:45.915612 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:45.915954 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:45.916670 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:45.916703 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:45.919389 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:45.922245 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:45.922254 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:45.922268 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:45.922975 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:45.923599 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:45.923604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:45.923608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:45.923802 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:45.924465 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:45.924471 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:45.924779 (+ 308us) client_negotiation.cc:769] Sending connection context
0807 19:37:45.925001 (+ 222us) client_negotiation.cc:240] Negotiation successful
0807 19:37:45.925355 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":398,"threads_started":1}
I20260807 19:37:45.927945 17960 heartbeater.cc:347] Connected to a master server at 127.14.16.126:36847
I20260807 19:37:45.928248 17960 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
I20260807 19:37:45.929023 17960 heartbeater.cc:511] Master 127.14.16.126:36847 requested a full tablet report, sending...
I20260807 19:37:45.930536 17627 ts_manager.cc:194] Registered new tserver with Master: 21108e6dec7f4e9d9a4d21ee710a549b (127.14.16.66:39777)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
I20260807 19:37:45.931488 17627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:54451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:37:45.956702 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:36847
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:46.286774 17965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:46.287065 17965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:46.287163 17965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:46.294667 17965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:46.294791 17965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:46.294910 17965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:37:46.304580 17965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:36847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:46.305872 17965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:46.307135 17965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:46.319967 17971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:46.319994 17974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:46.319926 17972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:46.321054 17965 server_base.cc:1034] running on GCE node
I20260807 19:37:46.321743 17965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:46.322723 17965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:46.323968 17965 hybrid_clock.cc:648] HybridClock initialized: now 1786131466323894 us; error 62 us; skew 500 ppm
I20260807 19:37:46.324297 17965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:46 dist-test-slave-07c2 krb5kdc[17582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131466, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:46.327527 17965 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:37:46.329020 17965 webserver.cc:460] Webserver started at http://127.14.16.67:41313/ using document root <none> and password file <none>
I20260807 19:37:46.329685 17965 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:46.329834 17965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:46.330147 17965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:46.333217 17965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "516c004bc21b42de8a148b0bb5b8fcad"
format_stamp: "Formatted at 2026-08-07 19:37:46 on dist-test-slave-07c2"
server_key: "f8fc818fc1ca4792811b3b35d01f9e3f"
server_key_iv: "eb1bf40ffc4df9b1cad95b0b0a650741"
server_key_version: "encryptionkey@0"
I20260807 19:37:46.334007 17965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "516c004bc21b42de8a148b0bb5b8fcad"
format_stamp: "Formatted at 2026-08-07 19:37:46 on dist-test-slave-07c2"
server_key: "f8fc818fc1ca4792811b3b35d01f9e3f"
server_key_iv: "eb1bf40ffc4df9b1cad95b0b0a650741"
server_key_version: "encryptionkey@0"
I20260807 19:37:46.339237 17965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:37:46.342896 17983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:46.344233 17965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:37:46.344421 17965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "516c004bc21b42de8a148b0bb5b8fcad"
format_stamp: "Formatted at 2026-08-07 19:37:46 on dist-test-slave-07c2"
server_key: "f8fc818fc1ca4792811b3b35d01f9e3f"
server_key_iv: "eb1bf40ffc4df9b1cad95b0b0a650741"
server_key_version: "encryptionkey@0"
I20260807 19:37:46.344616 17965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:46.369897 17965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:46.374217 17965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:46.374652 17965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:46.375684 17965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:46.377004 17965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:46.377120 17965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:46.377272 17965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:46.377343 17965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:46.411705 17965 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:38493
I20260807 19:37:46.411794 18098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:38493 every 8 connection(s)
I20260807 19:37:46.412961 17965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:37:46.416973 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 17965
I20260807 19:37:46.417156 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:37:46.417634 14401 external_mini_cluster.cc:1428] Setting key d2d6aba5ebe06db8ab31111ffa35b415
Aug 07 19:37:46 dist-test-slave-07c2 krb5kdc[17582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131466, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:46.428877 17823 heartbeater.cc:503] Master 127.14.16.126:36847 was elected leader, sending a full tablet report...
I20260807 19:37:46.433197 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:46.415539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:54119 (local address 127.14.16.126:36847)
0807 19:37:46.415751 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:46.415758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:46.416878 (+ 1120us) server_negotiation.cc:407] Connection header received
0807 19:37:46.418963 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:46.418974 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:46.419067 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:46.419153 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:46.420979 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:46.421736 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:46.423202 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:46.423711 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:46.423808 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:46.427039 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:46.427048 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:46.427055 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:46.427103 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:46.430002 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:46.430805 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:46.430809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:46.430811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:46.430936 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:46.431438 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:46.431443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:46.431446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:46.431775 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:46.431934 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:46.432638 (+ 704us) server_negotiation.cc:299] Negotiation successful
0807 19:37:46.432856 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:37:46.434125 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:46.416011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:36847 (local address 127.14.16.67:54119)
0807 19:37:46.416712 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:46.416765 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:46.418656 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:46.419457 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:46.419470 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:46.420106 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:46.420801 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:46.420830 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:46.421864 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:46.421868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:46.423059 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:46.423071 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:46.423896 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:46.423901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:46.424088 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:46.424700 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:46.424727 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:46.426813 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:46.430141 (+ 3328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:46.430149 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:46.430165 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:46.430688 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:46.431049 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:46.431053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:46.431056 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:46.431317 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:46.431916 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:46.431924 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:46.432251 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 19:37:46.432507 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 19:37:46.433019 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":278,"threads_started":1}
I20260807 19:37:46.435407 18099 heartbeater.cc:347] Connected to a master server at 127.14.16.126:36847
I20260807 19:37:46.435712 18099 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:46.436445 18099 heartbeater.cc:511] Master 127.14.16.126:36847 requested a full tablet report, sending...
I20260807 19:37:46.437861 17627 ts_manager.cc:194] Registered new tserver with Master: 516c004bc21b42de8a148b0bb5b8fcad (127.14.16.67:38493)
I20260807 19:37:46.438649 17627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:54119
I20260807 19:37:46.445312 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:37:46 dist-test-slave-07c2 krb5kdc[17582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131466, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 19:37:46 dist-test-slave-07c2 krb5kdc[17582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131466, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:46.497470 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:46.484151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34974 (local address 127.14.16.126:36847)
0807 19:37:46.484409 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:46.484419 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:46.484784 (+ 365us) server_negotiation.cc:407] Connection header received
0807 19:37:46.484937 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:46.484941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:46.485008 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:46.485097 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:46.486481 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:46.487187 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:46.488453 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:46.488861 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:46.488957 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:46.492800 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:46.492805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:46.492809 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:46.492843 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:46.495224 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:46.495864 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:46.495868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:46.495870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:46.495999 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:46.496329 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:46.496332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:46.496334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:46.496662 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:46.496862 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:46.497082 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 19:37:46.497248 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 19:37:46.498104 17627 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34974
I20260807 19:37:46.499658 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17691
Aug 07 19:37:46 dist-test-slave-07c2 krb5kdc[17582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131466, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:37:46.516426 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:46.500764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37502 (local address 127.14.16.65:37493)
0807 19:37:46.501415 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:46.501422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:46.501444 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:37:46.501498 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:46.501504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:46.501698 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:46.501864 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:46.503275 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:46.504365 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:46.505594 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:46.506171 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:46.506325 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:46.510379 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:46.510391 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:46.510405 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:46.510441 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:46.513488 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:46.514392 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:46.514399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:46.514402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:46.514531 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:46.514917 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:46.514920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:46.514922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:46.515276 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:46.515683 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:46.515899 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 19:37:46.516123 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":311,"threads_started":1}
W20260807 19:37:46.517771 17818 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37502
W20260807 19:37:46.518810 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17691: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:37:46.519003 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 17691
I20260807 19:37:46.538262 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17830
Aug 07 19:37:46 dist-test-slave-07c2 krb5kdc[17582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131466, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:37:46.554872 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:46.539102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58536 (local address 127.14.16.66:39777)
0807 19:37:46.539744 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:46.539751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:46.539780 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:37:46.539868 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:46.539874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:46.540080 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:46.540212 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:46.541498 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:46.542521 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:46.543809 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:46.544365 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:46.544462 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:46.548826 (+ 4364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:46.548837 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:46.548853 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:46.548911 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:46.552273 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:46.552905 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:46.552912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:46.552915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:46.553061 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:46.553388 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:46.553391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:46.553393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:46.553733 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:46.553982 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:46.554232 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 19:37:46.554432 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":364,"threads_started":1}
W20260807 19:37:46.556243 17955 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58536
W20260807 19:37:46.557379 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17830: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:37:46.557533 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 17830
I20260807 19:37:46.576551 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17965
Aug 07 19:37:46 dist-test-slave-07c2 krb5kdc[17582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131466, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:37:46.593393 18114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:46.577699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36618 (local address 127.14.16.67:38493)
0807 19:37:46.578475 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:46.578482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:46.578511 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:37:46.578573 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:46.578579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:46.578785 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:46.578913 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:46.580382 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:46.581470 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:46.582659 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:46.583225 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:46.583471 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:46.587890 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:46.587901 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:46.587915 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:46.587950 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:46.590714 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:46.591418 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:46.591425 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:46.591428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:46.591553 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:46.591908 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:46.591912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:46.591914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:46.592303 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:46.592568 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:46.592755 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 19:37:46.592965 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":462,"threads_started":1}
W20260807 19:37:46.594664 18094 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36618
W20260807 19:37:46.595803 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17965: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:37:46.595992 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 17965
I20260807 19:37:46.614120 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 17595
I20260807 19:37:46.615272 17660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:46.721658 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 17595
2026-08-07T19:37:46Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3696 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:37:46 dist-test-slave-07c2 krb5kdc[18118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:37:46 dist-test-slave-07c2 krb5kdc[18118](info): set up 2 sockets
Aug 07 19:37:46 dist-test-slave-07c2 krb5kdc[18118](info): commencing operation
krb5kdc: starting...
W20260807 19:37:47.832610 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:37:47 dist-test-slave-07c2 krb5kdc[18118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:37:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:37:48.062909 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:39777
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:39777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:48.384945 18131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:48.385283 18131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:48.385444 18131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:48.392539 18131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:48.392678 18131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:48.392781 18131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:48.392887 18131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:48.392975 18131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:48.402448 18131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:39777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:39777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:48.403717 18131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:48.405018 18131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:48.416508 18137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:48.416559 18138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:48.416471 18140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:48.417593 18131 server_base.cc:1034] running on GCE node
I20260807 19:37:48.418320 18131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:48.419624 18131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:48.420826 18131 hybrid_clock.cc:648] HybridClock initialized: now 1786131468420786 us; error 26 us; skew 500 ppm
I20260807 19:37:48.421108 18131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:48 dist-test-slave-07c2 krb5kdc[18118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131468, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:48.429661 18131 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:37:48.431119 18131 webserver.cc:460] Webserver started at http://127.14.16.126:41405/ using document root <none> and password file <none>
I20260807 19:37:48.431823 18131 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:48.431922 18131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:48.432191 18131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:48.435139 18131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "423151e6a2384794b82f701c7f9f866c"
format_stamp: "Formatted at 2026-08-07 19:37:48 on dist-test-slave-07c2"
server_key: "e50cc344260926b33bf30654a791f018"
server_key_iv: "aea9ada0aa9d9f9a8c6d033c273c795b"
server_key_version: "encryptionkey@0"
I20260807 19:37:48.435946 18131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "423151e6a2384794b82f701c7f9f866c"
format_stamp: "Formatted at 2026-08-07 19:37:48 on dist-test-slave-07c2"
server_key: "e50cc344260926b33bf30654a791f018"
server_key_iv: "aea9ada0aa9d9f9a8c6d033c273c795b"
server_key_version: "encryptionkey@0"
I20260807 19:37:48.440940 18131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 19:37:48.444386 18147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:48.445704 18131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:37:48.445883 18131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "423151e6a2384794b82f701c7f9f866c"
format_stamp: "Formatted at 2026-08-07 19:37:48 on dist-test-slave-07c2"
server_key: "e50cc344260926b33bf30654a791f018"
server_key_iv: "aea9ada0aa9d9f9a8c6d033c273c795b"
server_key_version: "encryptionkey@0"
I20260807 19:37:48.446082 18131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:48.461719 18131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:48.465483 18131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:48.465777 18131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:48.484175 18131 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:39777
I20260807 19:37:48.484198 18199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:39777 every 8 connection(s)
I20260807 19:37:48.485582 18131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:48.489702 18200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:48.493971 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 18131
I20260807 19:37:48.494213 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:48.495016 14401 external_mini_cluster.cc:1428] Setting key cf26e96e0c230c9911d92c7e8dbbda32
I20260807 19:37:48.501585 18200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c: Bootstrap starting.
I20260807 19:37:48.505268 18200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:48.506464 18200 log.cc:824] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:48.510113 18200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c: No bootstrap required, opened a new log
Aug 07 19:37:48 dist-test-slave-07c2 krb5kdc[18118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:48.518532 18200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "423151e6a2384794b82f701c7f9f866c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39777 } }
I20260807 19:37:48.518955 18200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:48.519093 18200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 423151e6a2384794b82f701c7f9f866c, State: Initialized, Role: FOLLOWER
I20260807 19:37:48.519769 18200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "423151e6a2384794b82f701c7f9f866c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39777 } }
I20260807 19:37:48.520040 18200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:48.520200 18200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:48.520396 18200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:48.520895 18203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:48.497141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37556 (local address 127.14.16.126:39777)
0807 19:37:48.497952 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:48.497963 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:48.498009 (+ 46us) server_negotiation.cc:407] Connection header received
0807 19:37:48.500107 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:48.500129 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:48.500490 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:48.500818 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:48.502767 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:48.504180 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:48.506003 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:48.506782 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:48.506919 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:48.511983 (+ 5064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:48.511991 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:48.512006 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:48.512046 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:48.515751 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:48.516555 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:48.516566 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:48.516580 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:48.516792 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:48.517260 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:48.517268 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:48.517279 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:48.517906 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:48.518247 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:48.518780 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 19:37:48.519526 (+ 746us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":399,"threads_started":1}
I20260807 19:37:48.522907 18200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "423151e6a2384794b82f701c7f9f866c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39777 } }
I20260807 19:37:48.523478 18200 leader_election.cc:302] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 423151e6a2384794b82f701c7f9f866c; no voters:
I20260807 19:37:48.524201 18200 leader_election.cc:288] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:48.524370 18205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:48.525130 18205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [term 1 LEADER]: Becoming Leader. State: Replica: 423151e6a2384794b82f701c7f9f866c, State: Running, Role: LEADER
I20260807 19:37:48.525739 18205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "423151e6a2384794b82f701c7f9f866c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39777 } }
I20260807 19:37:48.526257 18200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:48.528748 18207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 423151e6a2384794b82f701c7f9f866c. Latest consensus state: current_term: 1 leader_uuid: "423151e6a2384794b82f701c7f9f866c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "423151e6a2384794b82f701c7f9f866c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39777 } } }
I20260807 19:37:48.529173 18207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:48.530052 18206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "423151e6a2384794b82f701c7f9f866c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "423151e6a2384794b82f701c7f9f866c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39777 } } }
I20260807 19:37:48.530454 18206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:48.533099 18210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:48.538118 18210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:48.547725 18210 catalog_manager.cc:1347] Generated new cluster ID: d09f08eebc834521accd29bbcd2a0604
I20260807 19:37:48.547870 18210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:48.575359 18210 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:48.577481 18210 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:48.589291 18210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 423151e6a2384794b82f701c7f9f866c: Generated new TSK 0
I20260807 19:37:48.590423 18210 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:37:48.654724 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39777
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:48.986822 18226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:48.987142 18226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:48.987373 18226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:48.994599 18226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:48.994745 18226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:48.994940 18226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:37:49.004540 18226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:49.005802 18226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:49.007092 18226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:49.020254 18234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:49.020452 18232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:49.020895 18231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:49.021724 18226 server_base.cc:1034] running on GCE node
I20260807 19:37:49.022331 18226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:49.023417 18226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:49.024638 18226 hybrid_clock.cc:648] HybridClock initialized: now 1786131469024581 us; error 48 us; skew 500 ppm
I20260807 19:37:49.024943 18226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:49 dist-test-slave-07c2 krb5kdc[18118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131469, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:49.028201 18226 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:37:49.029613 18226 webserver.cc:460] Webserver started at http://127.14.16.65:45497/ using document root <none> and password file <none>
I20260807 19:37:49.030443 18226 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:49.030581 18226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:49.030983 18226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:49.033972 18226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "300b3e622ae341ff8a6598021315bac5"
format_stamp: "Formatted at 2026-08-07 19:37:49 on dist-test-slave-07c2"
server_key: "c7a31c48bfd9969a9eee8b267e9c5207"
server_key_iv: "1d7b1784d3aad4576ca08cc2c658dd1e"
server_key_version: "encryptionkey@0"
I20260807 19:37:49.034766 18226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "300b3e622ae341ff8a6598021315bac5"
format_stamp: "Formatted at 2026-08-07 19:37:49 on dist-test-slave-07c2"
server_key: "c7a31c48bfd9969a9eee8b267e9c5207"
server_key_iv: "1d7b1784d3aad4576ca08cc2c658dd1e"
server_key_version: "encryptionkey@0"
I20260807 19:37:49.040067 18226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 19:37:49.043529 18241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:49.044777 18226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:37:49.044957 18226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "300b3e622ae341ff8a6598021315bac5"
format_stamp: "Formatted at 2026-08-07 19:37:49 on dist-test-slave-07c2"
server_key: "c7a31c48bfd9969a9eee8b267e9c5207"
server_key_iv: "1d7b1784d3aad4576ca08cc2c658dd1e"
server_key_version: "encryptionkey@0"
I20260807 19:37:49.045164 18226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:49.061756 18226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:49.065588 18226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:49.065884 18226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:49.066862 18226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:49.068313 18226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:49.068387 18226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:49.068557 18226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:49.068622 18226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:49.102365 18226 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:42849
I20260807 19:37:49.102408 18355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:42849 every 8 connection(s)
I20260807 19:37:49.103744 18226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:37:49.108901 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 18226
I20260807 19:37:49.109127 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:37:49.109874 14401 external_mini_cluster.cc:1428] Setting key ed89366295f3bcb0b4c4a10c54b6782d
Aug 07 19:37:49 dist-test-slave-07c2 krb5kdc[18118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131469, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:49.125113 18358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:49.106267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:35111 (local address 127.14.16.126:39777)
0807 19:37:49.106784 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:49.106792 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:49.107748 (+ 956us) server_negotiation.cc:407] Connection header received
0807 19:37:49.109660 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:49.109667 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:49.109776 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:49.109908 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:49.111887 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:49.112960 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:49.114498 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:49.115207 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:49.115379 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:49.118719 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:49.118728 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:49.118735 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:49.118786 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:49.121839 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:49.122529 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:49.122537 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:49.122544 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:49.122691 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:49.123097 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:49.123105 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:49.123111 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:49.123511 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:49.123657 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:49.124364 (+ 707us) server_negotiation.cc:299] Negotiation successful
0807 19:37:49.124640 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":230,"threads_started":1}
I20260807 19:37:49.125743 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:49.106767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39777 (local address 127.14.16.65:35111)
0807 19:37:49.107585 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:49.107627 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:49.109389 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:49.110152 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:49.110160 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:49.110839 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:49.111702 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:49.111732 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:49.113127 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:49.113131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:49.114354 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:49.114365 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:49.115460 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:49.115465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:49.115659 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:49.116358 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:49.116380 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:49.118505 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:49.121961 (+ 3456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:49.121967 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:49.121979 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:49.122395 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:49.122796 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:49.122800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:49.122802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:49.122981 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:49.123644 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:49.123649 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:49.123951 (+ 302us) client_negotiation.cc:769] Sending connection context
0807 19:37:49.124182 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 19:37:49.124561 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":278,"threads_started":1}
I20260807 19:37:49.127507 18356 heartbeater.cc:347] Connected to a master server at 127.14.16.126:39777
I20260807 19:37:49.127842 18356 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:49.128572 18356 heartbeater.cc:511] Master 127.14.16.126:39777 requested a full tablet report, sending...
I20260807 19:37:49.130602 18164 ts_manager.cc:194] Registered new tserver with Master: 300b3e622ae341ff8a6598021315bac5 (127.14.16.65:42849)
I20260807 19:37:49.132102 18164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:35111
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:37:49.160727 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39777
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:49.489728 18362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:49.490005 18362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:49.490172 18362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:49.497503 18362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:49.497615 18362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:49.497825 18362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:37:49.507648 18362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:49.508978 18362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:49.510303 18362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:49.524317 18368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:49.524438 18367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:49.524410 18370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:49.525327 18362 server_base.cc:1034] running on GCE node
I20260807 19:37:49.525954 18362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:49.527050 18362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:49.528255 18362 hybrid_clock.cc:648] HybridClock initialized: now 1786131469528185 us; error 55 us; skew 500 ppm
I20260807 19:37:49.528590 18362 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:49 dist-test-slave-07c2 krb5kdc[18118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131469, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:49.540710 18362 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:37:49.542218 18362 webserver.cc:460] Webserver started at http://127.14.16.66:36855/ using document root <none> and password file <none>
I20260807 19:37:49.542935 18362 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:49.543092 18362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:49.543493 18362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:49.546571 18362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "d4844f8e1766401fb37dd6efd71b970e"
format_stamp: "Formatted at 2026-08-07 19:37:49 on dist-test-slave-07c2"
server_key: "98e7c704320074fc138e64d9820dc570"
server_key_iv: "3f14f7b2e7679b7e98800d04ca327992"
server_key_version: "encryptionkey@0"
I20260807 19:37:49.547461 18362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "d4844f8e1766401fb37dd6efd71b970e"
format_stamp: "Formatted at 2026-08-07 19:37:49 on dist-test-slave-07c2"
server_key: "98e7c704320074fc138e64d9820dc570"
server_key_iv: "3f14f7b2e7679b7e98800d04ca327992"
server_key_version: "encryptionkey@0"
I20260807 19:37:49.553032 18362 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:37:49.556778 18377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:49.558135 18362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:37:49.558326 18362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "d4844f8e1766401fb37dd6efd71b970e"
format_stamp: "Formatted at 2026-08-07 19:37:49 on dist-test-slave-07c2"
server_key: "98e7c704320074fc138e64d9820dc570"
server_key_iv: "3f14f7b2e7679b7e98800d04ca327992"
server_key_version: "encryptionkey@0"
I20260807 19:37:49.558548 18362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:49.575390 18362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:49.581040 18362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:49.581310 18362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:49.582201 18362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:49.583604 18362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:49.583680 18362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:49.583766 18362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:49.583815 18362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:49.622289 18362 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:41913
I20260807 19:37:49.622318 18492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:41913 every 8 connection(s)
I20260807 19:37:49.624189 18362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:37:49.631688 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 18362
I20260807 19:37:49.631901 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:37:49.632383 14401 external_mini_cluster.cc:1428] Setting key b2cded2e182a5ed639a44ef3a827ef5a
Aug 07 19:37:49 dist-test-slave-07c2 krb5kdc[18118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131469, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:49.653630 18496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:49.627593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:60935 (local address 127.14.16.126:39777)
0807 19:37:49.628317 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:49.628322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:49.629224 (+ 902us) server_negotiation.cc:407] Connection header received
0807 19:37:49.631089 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:49.631094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:49.631170 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:49.631333 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:49.633538 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:49.634404 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:49.636916 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:49.637669 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:49.637814 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:49.642898 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:49.642907 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:49.642913 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:49.642961 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:49.649541 (+ 6580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:49.650821 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:49.650826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:49.650830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:49.650975 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:49.651618 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:49.651623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:49.651627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:49.652021 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:49.652198 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:49.653015 (+ 817us) server_negotiation.cc:299] Negotiation successful
0807 19:37:49.653255 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":521,"threads_started":1}
I20260807 19:37:49.654706 18497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:49.628200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39777 (local address 127.14.16.66:60935)
0807 19:37:49.629080 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:49.629126 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:49.630850 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:49.631632 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:49.631646 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:49.632544 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:49.633363 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:49.633391 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:49.634606 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:49.634612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:49.636752 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:49.636767 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:49.637935 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:49.637945 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:49.638304 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:49.639151 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:49.639186 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:49.642600 (+ 3414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:49.649850 (+ 7250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:49.649863 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:49.649883 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:49.650679 (+ 796us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:49.651137 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:49.651142 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:49.651145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:49.651470 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:49.652164 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:49.652172 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:49.652626 (+ 454us) client_negotiation.cc:769] Sending connection context
0807 19:37:49.652902 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 19:37:49.653445 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":349,"threads_started":1}
I20260807 19:37:49.656318 18493 heartbeater.cc:347] Connected to a master server at 127.14.16.126:39777
I20260807 19:37:49.656659 18493 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:49.657392 18493 heartbeater.cc:511] Master 127.14.16.126:39777 requested a full tablet report, sending...
I20260807 19:37:49.658809 18164 ts_manager.cc:194] Registered new tserver with Master: d4844f8e1766401fb37dd6efd71b970e (127.14.16.66:41913)
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
I20260807 19:37:49.659744 18164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:60935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:37:49.687374 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39777
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:50.011637 18500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:50.011986 18500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:50.012172 18500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:50.019758 18500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:50.019913 18500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:50.020088 18500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:37:50.030087 18500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:50.031550 18500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:50.032879 18500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:50.046362 18505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:50.046337 18508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:50.046290 18506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:50.047082 18500 server_base.cc:1034] running on GCE node
I20260807 19:37:50.047708 18500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:50.048650 18500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:50.049911 18500 hybrid_clock.cc:648] HybridClock initialized: now 1786131470049859 us; error 51 us; skew 500 ppm
I20260807 19:37:50.050321 18500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:50 dist-test-slave-07c2 krb5kdc[18118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131470, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:50.053539 18500 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:37:50.055058 18500 webserver.cc:460] Webserver started at http://127.14.16.67:33541/ using document root <none> and password file <none>
I20260807 19:37:50.055806 18500 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:50.055964 18500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:50.056293 18500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:50.059154 18500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "f034f38bbf7d4732b6dfab3e773575ca"
format_stamp: "Formatted at 2026-08-07 19:37:50 on dist-test-slave-07c2"
server_key: "32a930b34bfc818cfc7c22f54ba0442f"
server_key_iv: "af0a68d2f2d9f2b85f8927f5fb89bbec"
server_key_version: "encryptionkey@0"
I20260807 19:37:50.060024 18500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "f034f38bbf7d4732b6dfab3e773575ca"
format_stamp: "Formatted at 2026-08-07 19:37:50 on dist-test-slave-07c2"
server_key: "32a930b34bfc818cfc7c22f54ba0442f"
server_key_iv: "af0a68d2f2d9f2b85f8927f5fb89bbec"
server_key_version: "encryptionkey@0"
I20260807 19:37:50.065192 18500 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 19:37:50.068840 18515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:50.070081 18500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:37:50.070269 18500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "f034f38bbf7d4732b6dfab3e773575ca"
format_stamp: "Formatted at 2026-08-07 19:37:50 on dist-test-slave-07c2"
server_key: "32a930b34bfc818cfc7c22f54ba0442f"
server_key_iv: "af0a68d2f2d9f2b85f8927f5fb89bbec"
server_key_version: "encryptionkey@0"
I20260807 19:37:50.070483 18500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:50.086311 18500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:50.090248 18500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:50.090541 18500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:50.091503 18500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:50.092779 18500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:50.092885 18500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:50.093040 18500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:50.093114 18500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:50.128233 18500 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:39037
I20260807 19:37:50.128261 18629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:39037 every 8 connection(s)
I20260807 19:37:50.129595 18500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:37:50.134939 18356 heartbeater.cc:503] Master 127.14.16.126:39777 was elected leader, sending a full tablet report...
I20260807 19:37:50.138566 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 18500
I20260807 19:37:50.138748 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:37:50.139182 14401 external_mini_cluster.cc:1428] Setting key 18831a9961d6aba6d65608df618a6e05
Aug 07 19:37:50 dist-test-slave-07c2 krb5kdc[18118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131470, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:50.149376 18496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:50.132272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:39075 (local address 127.14.16.126:39777)
0807 19:37:50.132471 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:50.132476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:50.133666 (+ 1190us) server_negotiation.cc:407] Connection header received
0807 19:37:50.135560 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:50.135567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:50.135652 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:50.135776 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:50.137506 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.138247 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.140005 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.140448 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.140586 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:50.143969 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:50.143975 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:50.143978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:50.144008 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:50.146431 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:50.147156 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:50.147161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:50.147163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:50.147338 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:50.147793 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:50.147798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:50.147801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:50.148071 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:50.148256 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:50.148871 (+ 615us) server_negotiation.cc:299] Negotiation successful
0807 19:37:50.149093 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:37:50.150462 18632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:50.132745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39777 (local address 127.14.16.67:39075)
0807 19:37:50.133521 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:50.133562 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:50.135213 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:50.136035 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:50.136047 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:50.136806 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:50.137353 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:50.137369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.138426 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.138430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:50.139855 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:50.139867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.140673 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.140681 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:50.140919 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:50.141590 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:50.141612 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:50.143770 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:50.146581 (+ 2811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:50.146589 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:50.146604 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:50.147019 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:50.147457 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:50.147462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:50.147466 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:50.147660 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:50.148230 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:50.148236 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:50.148549 (+ 313us) client_negotiation.cc:769] Sending connection context
0807 19:37:50.148772 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 19:37:50.149251 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":283,"threads_started":1}
I20260807 19:37:50.151933 18630 heartbeater.cc:347] Connected to a master server at 127.14.16.126:39777
I20260807 19:37:50.152231 18630 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:50.153214 18630 heartbeater.cc:511] Master 127.14.16.126:39777 requested a full tablet report, sending...
I20260807 19:37:50.154693 18164 ts_manager.cc:194] Registered new tserver with Master: f034f38bbf7d4732b6dfab3e773575ca (127.14.16.67:39037)
I20260807 19:37:50.155551 18164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:39075
I20260807 19:37:50.166303 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:37:50 dist-test-slave-07c2 krb5kdc[18118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:37:50 dist-test-slave-07c2 krb5kdc[18118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:37:50.221084 18637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:50.206465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52990 (local address 127.14.16.65:42849)
0807 19:37:50.207177 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:50.207185 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:50.207211 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:37:50.207275 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:50.207325 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:50.207551 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:50.207680 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:50.209056 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.210032 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.211211 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.211861 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.211955 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:50.215677 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:50.215687 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:50.215700 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:50.215733 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:50.218653 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:50.219235 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:50.219242 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:50.219245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:50.219417 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:50.219727 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:50.219730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:50.219732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:50.220053 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:50.220264 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:50.220514 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 19:37:50.220694 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":379,"threads_started":1}
Aug 07 19:37:50 dist-test-slave-07c2 krb5kdc[18118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:50.238641 18496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:50.225060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37562 (local address 127.14.16.126:39777)
0807 19:37:50.225275 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:50.225281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:50.225470 (+ 189us) server_negotiation.cc:407] Connection header received
0807 19:37:50.225662 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:50.225667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:50.225761 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:50.225866 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:50.227133 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.227936 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.229168 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.229585 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.229720 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:50.233520 (+ 3800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:50.233527 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:50.233531 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:50.233587 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:50.236058 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:50.236675 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:50.236681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:50.236684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:50.236826 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:50.237204 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:50.237209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:50.237213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:50.237517 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:50.237813 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:50.238092 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 19:37:50.238311 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 19:37:50.239195 18164 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37562
I20260807 19:37:50.240236 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 18226
I20260807 19:37:50.252303 18637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:50.241015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53002 (local address 127.14.16.65:42849)
0807 19:37:50.241235 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:50.241245 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:50.241508 (+ 263us) server_negotiation.cc:407] Connection header received
0807 19:37:50.241622 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:50.241626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:50.241687 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:50.241773 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:50.242886 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.243607 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.244772 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.245162 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.245256 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:50.247368 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:50.247379 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:50.247383 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:50.247410 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:50.249981 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:50.250681 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:50.250685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:50.250687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:50.250810 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:50.251126 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:50.251129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:50.251131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:50.251421 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:50.251589 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:50.251809 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 19:37:50.251962 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:37:50.252962 18351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:50.335180 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 18226
I20260807 19:37:50.353648 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 18362
Aug 07 19:37:50 dist-test-slave-07c2 krb5kdc[18118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:37:50.369896 18642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:50.354607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56512 (local address 127.14.16.66:41913)
0807 19:37:50.355124 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:50.355130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:50.355156 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:37:50.355215 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:50.355221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:50.355511 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:50.355641 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:50.356824 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.357854 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.359150 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.359829 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.359925 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:50.364092 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:50.364102 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:50.364115 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:50.364147 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:50.367022 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:50.367785 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:50.367792 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:50.367794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:50.367921 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:50.368301 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:50.368307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:50.368314 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:50.368653 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:50.368969 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:50.369325 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 19:37:50.369581 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":264,"threads_started":1}
I20260807 19:37:50.371109 18487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:50.453640 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 18362
I20260807 19:37:50.472316 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 18500
Aug 07 19:37:50 dist-test-slave-07c2 krb5kdc[18118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:37:50.489084 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:50.473454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38310 (local address 127.14.16.67:39037)
0807 19:37:50.474243 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:50.474250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:50.474278 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:37:50.474338 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:50.474345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:50.474555 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:50.474682 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:50.476075 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.477098 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.478347 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:50.478899 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:50.478995 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:50.483150 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:50.483161 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:50.483174 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:50.483208 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:50.486148 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:50.486935 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:50.486942 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:50.486944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:50.487061 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:50.487498 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:50.487505 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:50.487509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:50.487877 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:50.488116 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:50.488476 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 19:37:50.488696 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":472,"threads_started":1}
I20260807 19:37:50.490360 18625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:50.572978 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 18500
I20260807 19:37:50.591944 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 18131
I20260807 19:37:50.593264 18194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:50.699594 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 18131
2026-08-07T19:37:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3977 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:37:50 dist-test-slave-07c2 krb5kdc[18650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:37:50 dist-test-slave-07c2 krb5kdc[18650](info): set up 2 sockets
Aug 07 19:37:50 dist-test-slave-07c2 krb5kdc[18650](info): commencing operation
krb5kdc: starting...
W20260807 19:37:51.810473 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:37:51 dist-test-slave-07c2 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:37:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:37:52.036594 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:35379
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:35379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:52.365371 18663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:52.365664 18663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:52.365844 18663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:52.373117 18663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:52.373253 18663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:52.373348 18663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:52.373433 18663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:52.373493 18663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:52.383559 18663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:35379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:35379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:52.384883 18663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:52.386214 18663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:52.398577 18668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:52.398541 18671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:52.398625 18669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:52.399600 18663 server_base.cc:1034] running on GCE node
I20260807 19:37:52.400399 18663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:52.401814 18663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:52.403026 18663 hybrid_clock.cc:648] HybridClock initialized: now 1786131472402970 us; error 51 us; skew 500 ppm
I20260807 19:37:52.403502 18663 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:52 dist-test-slave-07c2 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131472, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:52.406716 18663 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:37:52.408105 18663 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:52.408239 18663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:52.408591 18663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:52.411674 18663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "9915e1536c964f0ba79b33aac8a9bda9"
format_stamp: "Formatted at 2026-08-07 19:37:52 on dist-test-slave-07c2"
server_key: "0f485b66980d69e8186e2700bee81ae3"
server_key_iv: "a229405290805ba554c912f9b69e457d"
server_key_version: "encryptionkey@0"
I20260807 19:37:52.412513 18663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "9915e1536c964f0ba79b33aac8a9bda9"
format_stamp: "Formatted at 2026-08-07 19:37:52 on dist-test-slave-07c2"
server_key: "0f485b66980d69e8186e2700bee81ae3"
server_key_iv: "a229405290805ba554c912f9b69e457d"
server_key_version: "encryptionkey@0"
I20260807 19:37:52.417661 18663 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:37:52.421272 18677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:52.422565 18663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:37:52.422778 18663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "9915e1536c964f0ba79b33aac8a9bda9"
format_stamp: "Formatted at 2026-08-07 19:37:52 on dist-test-slave-07c2"
server_key: "0f485b66980d69e8186e2700bee81ae3"
server_key_iv: "a229405290805ba554c912f9b69e457d"
server_key_version: "encryptionkey@0"
I20260807 19:37:52.423010 18663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:52.446599 18663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:52.450615 18663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:52.450896 18663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:52.469465 18663 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:35379
I20260807 19:37:52.469489 18729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:35379 every 8 connection(s)
I20260807 19:37:52.470352 18663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:52.473924 18730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:52.479452 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 18663
I20260807 19:37:52.479693 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:52.480507 14401 external_mini_cluster.cc:1428] Setting key 2562714cb22743c232440d2a94c230c9
I20260807 19:37:52.485383 18730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9: Bootstrap starting.
I20260807 19:37:52.489253 18730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:52.490545 18730 log.cc:824] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:52.493932 18730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9: No bootstrap required, opened a new log
Aug 07 19:37:52 dist-test-slave-07c2 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:52.502619 18730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9915e1536c964f0ba79b33aac8a9bda9" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 35379 } }
I20260807 19:37:52.503031 18730 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:52.503144 18730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9915e1536c964f0ba79b33aac8a9bda9, State: Initialized, Role: FOLLOWER
I20260807 19:37:52.504017 18730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9915e1536c964f0ba79b33aac8a9bda9" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 35379 } }
I20260807 19:37:52.504299 18730 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:52.504468 18730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:52.504686 18730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:52.505414 18733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:52.482696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43968 (local address 127.14.16.126:35379)
0807 19:37:52.483599 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:52.483611 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:52.483656 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:37:52.485361 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:52.485388 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:52.485829 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:52.486159 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:52.488160 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:52.489633 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:52.491566 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:52.492401 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:52.492543 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:52.497380 (+ 4837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:52.497391 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:52.497409 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:52.497443 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:52.500647 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:52.501352 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:52.501359 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:52.501365 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:52.501533 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:52.501953 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:52.501957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:52.501959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:52.502497 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:52.502811 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:52.503455 (+ 644us) server_negotiation.cc:299] Negotiation successful
0807 19:37:52.503898 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":395,"threads_started":1}
I20260807 19:37:52.507860 18730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9915e1536c964f0ba79b33aac8a9bda9" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 35379 } }
I20260807 19:37:52.508358 18730 leader_election.cc:302] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9915e1536c964f0ba79b33aac8a9bda9; no voters:
I20260807 19:37:52.508931 18730 leader_election.cc:288] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:52.509057 18736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:52.509836 18736 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [term 1 LEADER]: Becoming Leader. State: Replica: 9915e1536c964f0ba79b33aac8a9bda9, State: Running, Role: LEADER
I20260807 19:37:52.510380 18736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9915e1536c964f0ba79b33aac8a9bda9" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 35379 } }
I20260807 19:37:52.511404 18730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:52.513015 18737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9915e1536c964f0ba79b33aac8a9bda9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9915e1536c964f0ba79b33aac8a9bda9" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 35379 } } }
I20260807 19:37:52.513381 18737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:52.514252 18738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9915e1536c964f0ba79b33aac8a9bda9. Latest consensus state: current_term: 1 leader_uuid: "9915e1536c964f0ba79b33aac8a9bda9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9915e1536c964f0ba79b33aac8a9bda9" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 35379 } } }
I20260807 19:37:52.514801 18738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:52.521704 18742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:52.527170 18742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:52.538707 18742 catalog_manager.cc:1347] Generated new cluster ID: 14f6c4becdde4e55add2ef9c4145b871
I20260807 19:37:52.538841 18742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:52.560565 18742 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:52.561966 18742 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:52.572019 18742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9: Generated new TSK 0
I20260807 19:37:52.573079 18742 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:37:52.627779 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:35379
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:52.957926 18759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:52.958248 18759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:52.958349 18759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:52.965519 18759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:52.965622 18759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:52.965739 18759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:37:52.975250 18759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:35379
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:52.976533 18759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:52.977751 18759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:52.990492 18767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:52.990571 18765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:52.990543 18764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:52.991731 18759 server_base.cc:1034] running on GCE node
I20260807 19:37:52.992419 18759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:52.993359 18759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:52.994611 18759 hybrid_clock.cc:648] HybridClock initialized: now 1786131472994551 us; error 47 us; skew 500 ppm
I20260807 19:37:52.994963 18759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:52 dist-test-slave-07c2 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131472, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:52.998095 18759 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:37:52.999241 18759 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:52.999394 18759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:52.999665 18759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:53.002506 18759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "5e525a82a1e6454081a9fa5c5351994c"
format_stamp: "Formatted at 2026-08-07 19:37:52 on dist-test-slave-07c2"
server_key: "a9a274643b167372fff1d464ce4b93f0"
server_key_iv: "7fac03892a6b9eb65f715dde436dfba0"
server_key_version: "encryptionkey@0"
I20260807 19:37:53.003262 18759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "5e525a82a1e6454081a9fa5c5351994c"
format_stamp: "Formatted at 2026-08-07 19:37:52 on dist-test-slave-07c2"
server_key: "a9a274643b167372fff1d464ce4b93f0"
server_key_iv: "7fac03892a6b9eb65f715dde436dfba0"
server_key_version: "encryptionkey@0"
I20260807 19:37:53.008453 18759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:37:53.012039 18773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:53.013526 18759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:37:53.013756 18759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "5e525a82a1e6454081a9fa5c5351994c"
format_stamp: "Formatted at 2026-08-07 19:37:52 on dist-test-slave-07c2"
server_key: "a9a274643b167372fff1d464ce4b93f0"
server_key_iv: "7fac03892a6b9eb65f715dde436dfba0"
server_key_version: "encryptionkey@0"
I20260807 19:37:53.013991 18759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:53.031023 18759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:53.035068 18759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:53.035389 18759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:53.036341 18759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:53.037675 18759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:53.037786 18759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:53.037915 18759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:53.038018 18759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:53.072278 18759 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:38515
I20260807 19:37:53.072294 18891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:38515 every 8 connection(s)
I20260807 19:37:53.073191 18759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:37:53.080569 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 18759
I20260807 19:37:53.080816 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:37:53.081496 14401 external_mini_cluster.cc:1428] Setting key 83885e4e113c5958d5dbfe4ee461b9da
Aug 07 19:37:53 dist-test-slave-07c2 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131472, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:53.093411 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:53.075802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:48495 (local address 127.14.16.126:35379)
0807 19:37:53.076194 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:53.076198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:53.077130 (+ 932us) server_negotiation.cc:407] Connection header received
0807 19:37:53.078910 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:53.078916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:53.078982 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:53.079083 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:53.080903 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:53.081686 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:53.083329 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:53.083739 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:53.083878 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:53.087521 (+ 3643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:53.087526 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:53.087529 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:53.087566 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:53.090340 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:53.091085 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:53.091091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:53.091095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:53.091221 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:53.091677 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:53.091682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:53.091686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:53.091954 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:53.092114 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:53.092979 (+ 865us) server_negotiation.cc:299] Negotiation successful
0807 19:37:53.093150 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":188,"threads_started":1}
I20260807 19:37:53.094897 18895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:53.076283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:35379 (local address 127.14.16.65:48495)
0807 19:37:53.076961 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:53.077008 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:53.078639 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:53.079268 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:53.079276 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:53.079869 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:53.080643 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:53.080680 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:53.081849 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:53.081855 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:53.083142 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:53.083154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:53.083968 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:53.083975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:53.084191 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:53.084874 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:53.084897 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:53.087248 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:53.090495 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:53.090504 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:53.090519 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:53.090952 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:53.091360 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:53.091365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:53.091368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:53.091562 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:53.092095 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:53.092102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:53.092548 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 19:37:53.092852 (+ 304us) client_negotiation.cc:240] Negotiation successful
0807 19:37:53.093447 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":280,"threads_started":1}
I20260807 19:37:53.096560 18892 heartbeater.cc:347] Connected to a master server at 127.14.16.126:35379
I20260807 19:37:53.096881 18892 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:53.097534 18892 heartbeater.cc:511] Master 127.14.16.126:35379 requested a full tablet report, sending...
I20260807 19:37:53.099582 18694 ts_manager.cc:194] Registered new tserver with Master: 5e525a82a1e6454081a9fa5c5351994c (127.14.16.65:38515)
I20260807 19:37:53.101058 18694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:48495
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:37:53.132241 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:35379
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:53.460069 18898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:53.460371 18898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:53.460551 18898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:53.467833 18898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:53.467955 18898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:53.468138 18898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:37:53.482932 18898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:35379
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:53.484263 18898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:53.485571 18898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:53.498318 18906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:53.498391 18903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:53.498410 18904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:53.499228 18898 server_base.cc:1034] running on GCE node
I20260807 19:37:53.499881 18898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:53.500833 18898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:53.502064 18898 hybrid_clock.cc:648] HybridClock initialized: now 1786131473502002 us; error 57 us; skew 500 ppm
I20260807 19:37:53.502723 18898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:53 dist-test-slave-07c2 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131473, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:53.510221 18898 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:37:53.511683 18898 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:53.511826 18898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:53.512151 18898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:53.515205 18898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "917189f51cfb42fda84a369652aec8da"
format_stamp: "Formatted at 2026-08-07 19:37:53 on dist-test-slave-07c2"
server_key: "faf37f5df640d7a0db07eb4e6d2cf895"
server_key_iv: "0e9a231ec121b11c7c95cafbd424e01a"
server_key_version: "encryptionkey@0"
I20260807 19:37:53.516191 18898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "917189f51cfb42fda84a369652aec8da"
format_stamp: "Formatted at 2026-08-07 19:37:53 on dist-test-slave-07c2"
server_key: "faf37f5df640d7a0db07eb4e6d2cf895"
server_key_iv: "0e9a231ec121b11c7c95cafbd424e01a"
server_key_version: "encryptionkey@0"
I20260807 19:37:53.521445 18898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 19:37:53.525120 18912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:53.526664 18898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:37:53.526866 18898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "917189f51cfb42fda84a369652aec8da"
format_stamp: "Formatted at 2026-08-07 19:37:53 on dist-test-slave-07c2"
server_key: "faf37f5df640d7a0db07eb4e6d2cf895"
server_key_iv: "0e9a231ec121b11c7c95cafbd424e01a"
server_key_version: "encryptionkey@0"
I20260807 19:37:53.527128 18898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:53.555027 18898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:53.559590 18898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:53.559912 18898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:53.560922 18898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:53.562252 18898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:53.562376 18898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:53.562503 18898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:53.562605 18898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:53.601136 18898 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:45473
I20260807 19:37:53.601224 19028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:45473 every 8 connection(s)
I20260807 19:37:53.601987 18898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:37:53.603427 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 18898
I20260807 19:37:53.603628 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:37:53.604080 14401 external_mini_cluster.cc:1428] Setting key d0d95577dc6afd8af12dc1644706d2bf
Aug 07 19:37:53 dist-test-slave-07c2 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131473, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:53.623823 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:53.604598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:44927 (local address 127.14.16.126:35379)
0807 19:37:53.607373 (+ 2775us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:53.607378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:53.607397 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:37:53.608577 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:53.608583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:53.608663 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:53.608785 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:53.610250 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:53.611443 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:53.613116 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:53.613560 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:53.613696 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:53.617407 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:53.617415 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:53.617423 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:53.617473 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:53.620294 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:53.621231 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:53.621235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:53.621237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:53.621369 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:53.621883 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:53.621886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:53.621887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:53.622139 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:53.622255 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:53.623171 (+ 916us) server_negotiation.cc:299] Negotiation successful
0807 19:37:53.623398 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2621,"thread_start_us":2483,"threads_started":1}
I20260807 19:37:53.625296 19032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:53.605298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:35379 (local address 127.14.16.66:44927)
0807 19:37:53.606139 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:53.606178 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:53.608260 (+ 2082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:53.609003 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:53.609011 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:53.609537 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:53.610068 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:53.610084 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:53.611590 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:53.611594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:53.612936 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:53.612948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:53.613798 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:53.613805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:53.614022 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:53.614724 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:53.614746 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:53.617173 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:53.620441 (+ 3268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:53.620452 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:53.620471 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:53.621116 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:53.621479 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:53.621486 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:53.621492 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:53.621760 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:53.622271 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:53.622281 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:53.622774 (+ 493us) client_negotiation.cc:769] Sending connection context
0807 19:37:53.623085 (+ 311us) client_negotiation.cc:240] Negotiation successful
0807 19:37:53.623761 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":380,"threads_started":1}
I20260807 19:37:53.626844 19029 heartbeater.cc:347] Connected to a master server at 127.14.16.126:35379
I20260807 19:37:53.627177 19029 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:53.628209 19029 heartbeater.cc:511] Master 127.14.16.126:35379 requested a full tablet report, sending...
I20260807 19:37:53.629760 18694 ts_manager.cc:194] Registered new tserver with Master: 917189f51cfb42fda84a369652aec8da (127.14.16.66:45473)
I20260807 19:37:53.630532 18694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:44927
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:37:53.660837 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:35379
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:53.987198 19035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:53.987483 19035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:53.987582 19035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:53.994649 19035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:53.994757 19035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:53.994871 19035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:37:54.004760 19035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:35379
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:54.006038 19035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:54.007333 19035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:54.020511 19040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:54.020452 19043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:54.020411 19041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:54.021337 19035 server_base.cc:1034] running on GCE node
I20260807 19:37:54.021998 19035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:54.023052 19035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:54.024318 19035 hybrid_clock.cc:648] HybridClock initialized: now 1786131474024245 us; error 81 us; skew 500 ppm
I20260807 19:37:54.024663 19035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:54 dist-test-slave-07c2 krb5kdc[18650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131474, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:54.027935 19035 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:37:54.029234 19035 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:54.029362 19035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:54.029708 19035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:54.032655 19035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "d2aff676a6a54a518d6ed8b191a13e93"
format_stamp: "Formatted at 2026-08-07 19:37:54 on dist-test-slave-07c2"
server_key: "6bd90ec0a102a233e169694110854790"
server_key_iv: "45f02abbcf5dcb70eab62fa07dce6d79"
server_key_version: "encryptionkey@0"
I20260807 19:37:54.033401 19035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "d2aff676a6a54a518d6ed8b191a13e93"
format_stamp: "Formatted at 2026-08-07 19:37:54 on dist-test-slave-07c2"
server_key: "6bd90ec0a102a233e169694110854790"
server_key_iv: "45f02abbcf5dcb70eab62fa07dce6d79"
server_key_version: "encryptionkey@0"
I20260807 19:37:54.038394 19035 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:37:54.041896 19049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:54.043183 19035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:37:54.043398 19035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "d2aff676a6a54a518d6ed8b191a13e93"
format_stamp: "Formatted at 2026-08-07 19:37:54 on dist-test-slave-07c2"
server_key: "6bd90ec0a102a233e169694110854790"
server_key_iv: "45f02abbcf5dcb70eab62fa07dce6d79"
server_key_version: "encryptionkey@0"
I20260807 19:37:54.043645 19035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:54.068252 19035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:54.072319 19035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:54.072599 19035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:54.073611 19035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:54.074973 19035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:54.075088 19035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:54.075209 19035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:54.075320 19035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:54.103950 18892 heartbeater.cc:503] Master 127.14.16.126:35379 was elected leader, sending a full tablet report...
I20260807 19:37:54.109283 19035 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:44763
I20260807 19:37:54.109368 19165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:44763 every 8 connection(s)
I20260807 19:37:54.110152 19035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:37:54.112718 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 19035
I20260807 19:37:54.112957 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:37:54.113373 14401 external_mini_cluster.cc:1428] Setting key 41f324ea8b288819cb43436b3aaf6dba
Aug 07 19:37:54 dist-test-slave-07c2 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131474, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:37:54.130245 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.112580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:57609 (local address 127.14.16.126:35379)
0807 19:37:54.112802 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:54.112809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:54.114233 (+ 1424us) server_negotiation.cc:407] Connection header received
0807 19:37:54.116365 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:54.116371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:54.116463 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:54.116542 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:54.118474 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.119218 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.120737 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.121123 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.121213 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.124605 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:54.124610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:54.124613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:54.124646 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:54.127257 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:54.128104 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:54.128109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:54.128112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:54.128276 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:54.128742 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:54.128747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:54.128750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:54.129082 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:54.129199 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:54.129822 (+ 623us) server_negotiation.cc:299] Negotiation successful
0807 19:37:54.129993 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:37:54.131366 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.113237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:35379 (local address 127.14.16.67:57609)
0807 19:37:54.114066 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.114112 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:54.116101 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:54.116796 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:54.116809 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:54.117613 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:54.118319 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:54.118338 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.119406 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.119410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:54.120604 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:54.120616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.121316 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.121321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:54.121493 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.122083 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:54.122105 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:54.124421 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:54.127425 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:54.127434 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:54.127450 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:54.127987 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:54.128395 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:54.128399 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:54.128401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:54.128577 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:54.129196 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:54.129201 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:54.129507 (+ 306us) client_negotiation.cc:769] Sending connection context
0807 19:37:54.129730 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 19:37:54.130095 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":390,"threads_started":1}
I20260807 19:37:54.132712 19166 heartbeater.cc:347] Connected to a master server at 127.14.16.126:35379
I20260807 19:37:54.133024 19166 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:54.133646 19166 heartbeater.cc:511] Master 127.14.16.126:35379 requested a full tablet report, sending...
I20260807 19:37:54.134964 18694 ts_manager.cc:194] Registered new tserver with Master: d2aff676a6a54a518d6ed8b191a13e93 (127.14.16.67:44763)
I20260807 19:37:54.135774 18694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:57609
I20260807 19:37:54.140991 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:37:54.157552 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.145378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43976 (local address 127.14.16.126:35379)
0807 19:37:54.145586 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:54.145594 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:54.146094 (+ 500us) server_negotiation.cc:407] Connection header received
0807 19:37:54.146227 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:54.146232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:54.146291 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:54.146369 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:54.148258 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.149051 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.150210 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.150584 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.150678 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.152890 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:54.152894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:54.152897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:54.152924 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:54.155212 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:54.155857 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:54.155861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:54.155863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:54.155988 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:54.156301 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:54.156304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:54.156305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:54.156591 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:54.156784 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:54.157032 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 19:37:54.157181 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:37:54.161368 18694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43976:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:37:54.168107 18694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:37:54.191821 19178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.179753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:45473 (local address 127.0.0.1:58846)
0807 19:37:54.180406 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.180461 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:54.180617 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:54.181680 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:54.181688 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:54.181714 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:54.182082 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:54.182097 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.184611 (+ 2514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.184620 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:54.187723 (+ 3103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:54.187743 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.190597 (+ 2854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.190607 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:54.190910 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.190987 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 19:37:54.191150 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 19:37:54.191429 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":273,"threads_started":1}
I20260807 19:37:54.192633 19179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.179960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58846 (local address 127.14.16.66:45473)
0807 19:37:54.180505 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:54.180514 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:54.180544 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:37:54.180894 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:54.180903 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:54.181174 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:54.181374 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:54.182304 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.184405 (+ 2101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.187959 (+ 3554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.189968 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.190143 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.190426 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:54.192079 (+ 1653us) server_negotiation.cc:299] Negotiation successful
0807 19:37:54.192295 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":299,"threads_started":1}
I20260807 19:37:54.196599 18962 tablet_service.cc:1467] Processing CreateTablet for tablet ccb96637d1f44158af1bb79620056512 (DEFAULT_TABLE table=test-table [id=4d57a41fd0ee4b7f96d88902aa4b4aec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:54.198036 18962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccb96637d1f44158af1bb79620056512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:54.201689 19182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.182196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:44763 (local address 127.0.0.1:37548)
0807 19:37:54.191668 (+ 9472us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.191691 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:54.191830 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:54.194617 (+ 2787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:54.194622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:54.194648 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:54.195070 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:54.195083 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.197213 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.197219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:54.199172 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:54.199187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.200834 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.200841 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:54.201026 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.201061 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:37:54.201160 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 19:37:54.201299 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9345,"mutex_wait_us":28,"thread_start_us":9725,"threads_started":1}
I20260807 19:37:54.201969 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.182431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37548 (local address 127.14.16.67:44763)
0807 19:37:54.194042 (+ 11611us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:54.194053 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:54.194086 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:37:54.194188 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:54.194197 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:54.194420 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:54.194624 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:54.195355 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.197038 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.199411 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.200608 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.200752 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.200984 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:54.201221 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 19:37:54.201395 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11432,"thread_start_us":11818,"threads_started":1}
I20260807 19:37:54.203428 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.180830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:38515 (local address 127.0.0.1:32846)
0807 19:37:54.181366 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.181391 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:54.181544 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:54.194031 (+ 12487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:54.194035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:54.194064 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:54.194969 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:54.194989 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.197354 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.197362 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:54.199824 (+ 2462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:54.199840 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.201939 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.201945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:54.202713 (+ 768us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.202751 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:37:54.202854 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 19:37:54.202997 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":4,"thread_start_us":281,"threads_started":1}
I20260807 19:37:54.203876 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.181179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32846 (local address 127.14.16.65:38515)
0807 19:37:54.193140 (+ 11961us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:54.193149 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:54.193179 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:37:54.193283 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:54.193290 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:54.193537 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:54.193743 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:54.195205 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.197189 (+ 1984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.200003 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.201429 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.201569 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.201780 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:54.203136 (+ 1356us) server_negotiation.cc:299] Negotiation successful
0807 19:37:54.203339 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11788,"thread_start_us":12021,"threads_started":1}
I20260807 19:37:54.205641 19101 tablet_service.cc:1467] Processing CreateTablet for tablet ccb96637d1f44158af1bb79620056512 (DEFAULT_TABLE table=test-table [id=4d57a41fd0ee4b7f96d88902aa4b4aec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:54.206918 19101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccb96637d1f44158af1bb79620056512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:54.207317 18822 tablet_service.cc:1467] Processing CreateTablet for tablet ccb96637d1f44158af1bb79620056512 (DEFAULT_TABLE table=test-table [id=4d57a41fd0ee4b7f96d88902aa4b4aec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:54.209105 18822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccb96637d1f44158af1bb79620056512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:54.213445 19184 tablet_bootstrap.cc:492] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da: Bootstrap starting.
I20260807 19:37:54.217576 19184 tablet_bootstrap.cc:654] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:54.218959 19184 log.cc:824] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:54.221122 19186 tablet_bootstrap.cc:492] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93: Bootstrap starting.
I20260807 19:37:54.222142 19184 tablet_bootstrap.cc:492] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da: No bootstrap required, opened a new log
I20260807 19:37:54.222443 19184 ts_tablet_manager.cc:1397] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260807 19:37:54.224694 19186 tablet_bootstrap.cc:654] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:54.225893 19186 log.cc:824] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:54.227413 19187 tablet_bootstrap.cc:492] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c: Bootstrap starting.
I20260807 19:37:54.229209 19186 tablet_bootstrap.cc:492] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93: No bootstrap required, opened a new log
I20260807 19:37:54.229513 19186 ts_tablet_manager.cc:1397] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:37:54.233076 19184 raft_consensus.cc:348] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917189f51cfb42fda84a369652aec8da" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45473 } } peers { permanent_uuid: "5e525a82a1e6454081a9fa5c5351994c" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38515 } } peers { permanent_uuid: "d2aff676a6a54a518d6ed8b191a13e93" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44763 } }
I20260807 19:37:54.233536 19184 raft_consensus.cc:374] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:54.233644 19184 raft_consensus.cc:729] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 917189f51cfb42fda84a369652aec8da, State: Initialized, Role: FOLLOWER
I20260807 19:37:54.234249 19184 consensus_queue.cc:260] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917189f51cfb42fda84a369652aec8da" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45473 } } peers { permanent_uuid: "5e525a82a1e6454081a9fa5c5351994c" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38515 } } peers { permanent_uuid: "d2aff676a6a54a518d6ed8b191a13e93" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44763 } }
I20260807 19:37:54.235411 19029 heartbeater.cc:503] Master 127.14.16.126:35379 was elected leader, sending a full tablet report...
I20260807 19:37:54.236054 19184 ts_tablet_manager.cc:1428] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260807 19:37:54.238394 19187 tablet_bootstrap.cc:654] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:54.238891 19186 raft_consensus.cc:348] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917189f51cfb42fda84a369652aec8da" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45473 } } peers { permanent_uuid: "5e525a82a1e6454081a9fa5c5351994c" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38515 } } peers { permanent_uuid: "d2aff676a6a54a518d6ed8b191a13e93" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44763 } }
I20260807 19:37:54.239243 19186 raft_consensus.cc:374] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:54.239390 19186 raft_consensus.cc:729] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2aff676a6a54a518d6ed8b191a13e93, State: Initialized, Role: FOLLOWER
I20260807 19:37:54.239804 19187 log.cc:824] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:54.239996 19186 consensus_queue.cc:260] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917189f51cfb42fda84a369652aec8da" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45473 } } peers { permanent_uuid: "5e525a82a1e6454081a9fa5c5351994c" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38515 } } peers { permanent_uuid: "d2aff676a6a54a518d6ed8b191a13e93" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44763 } }
I20260807 19:37:54.241870 19186 ts_tablet_manager.cc:1428] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260807 19:37:54.242452 19166 heartbeater.cc:503] Master 127.14.16.126:35379 was elected leader, sending a full tablet report...
I20260807 19:37:54.243121 19187 tablet_bootstrap.cc:492] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c: No bootstrap required, opened a new log
I20260807 19:37:54.243463 19187 ts_tablet_manager.cc:1397] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.003s
I20260807 19:37:54.250916 19187 raft_consensus.cc:348] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917189f51cfb42fda84a369652aec8da" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45473 } } peers { permanent_uuid: "5e525a82a1e6454081a9fa5c5351994c" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38515 } } peers { permanent_uuid: "d2aff676a6a54a518d6ed8b191a13e93" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44763 } }
I20260807 19:37:54.251250 19187 raft_consensus.cc:374] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:54.251413 19187 raft_consensus.cc:729] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e525a82a1e6454081a9fa5c5351994c, State: Initialized, Role: FOLLOWER
I20260807 19:37:54.251896 19187 consensus_queue.cc:260] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917189f51cfb42fda84a369652aec8da" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45473 } } peers { permanent_uuid: "5e525a82a1e6454081a9fa5c5351994c" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38515 } } peers { permanent_uuid: "d2aff676a6a54a518d6ed8b191a13e93" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44763 } }
I20260807 19:37:54.253240 19187 ts_tablet_manager.cc:1428] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260807 19:37:54.305300 19192 raft_consensus.cc:482] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:54.305605 19192 raft_consensus.cc:504] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917189f51cfb42fda84a369652aec8da" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45473 } } peers { permanent_uuid: "5e525a82a1e6454081a9fa5c5351994c" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38515 } } peers { permanent_uuid: "d2aff676a6a54a518d6ed8b191a13e93" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44763 } }
I20260807 19:37:54.307359 19192 leader_election.cc:288] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 917189f51cfb42fda84a369652aec8da (127.14.16.66:45473), d2aff676a6a54a518d6ed8b191a13e93 (127.14.16.67:44763)
I20260807 19:37:54.315454 19179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.307842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:44523 (local address 127.14.16.66:45473)
0807 19:37:54.308116 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:54.308123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:54.308472 (+ 349us) server_negotiation.cc:407] Connection header received
0807 19:37:54.308762 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:54.308771 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:54.308882 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:54.308982 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:54.309684 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.310718 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.313233 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.314432 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.314598 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.314678 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:54.315045 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 19:37:54.315155 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:37:54.315696 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.308517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:44763 (local address 127.14.16.65:42565)
0807 19:37:54.309205 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.309229 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:54.309366 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:54.309701 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:54.309704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:54.309723 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:54.309984 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:54.309991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.311194 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.311199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:54.313291 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:54.313301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.314967 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.314976 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:54.315137 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.315165 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 19:37:54.315255 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 19:37:54.315416 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":320,"threads_started":1}
I20260807 19:37:54.315708 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.308693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:42565 (local address 127.14.16.67:44763)
0807 19:37:54.308962 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:54.308968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:54.309334 (+ 366us) server_negotiation.cc:407] Connection header received
0807 19:37:54.309494 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:54.309498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:54.309556 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:54.309664 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:37:54.310133 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.311067 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.313431 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.314695 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.314876 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.314951 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:54.315332 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 19:37:54.315465 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260807 19:37:54.315936 19193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.307711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:45473 (local address 127.14.16.65:44523)
0807 19:37:54.308354 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.308371 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:54.308570 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:54.309063 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:54.309067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:54.309095 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:37:54.309540 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:54.309550 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.310842 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.310846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:54.313073 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:54.313088 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.314658 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.314663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:54.314862 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.314913 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 19:37:54.315012 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 19:37:54.315144 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":358,"threads_started":1}
I20260807 19:37:54.316792 19121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccb96637d1f44158af1bb79620056512" candidate_uuid: "5e525a82a1e6454081a9fa5c5351994c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2aff676a6a54a518d6ed8b191a13e93" is_pre_election: true
I20260807 19:37:54.316825 18983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccb96637d1f44158af1bb79620056512" candidate_uuid: "5e525a82a1e6454081a9fa5c5351994c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917189f51cfb42fda84a369652aec8da" is_pre_election: true
I20260807 19:37:54.317341 18983 raft_consensus.cc:2454] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e525a82a1e6454081a9fa5c5351994c in term 0.
I20260807 19:37:54.317349 19121 raft_consensus.cc:2454] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e525a82a1e6454081a9fa5c5351994c in term 0.
I20260807 19:37:54.318025 18774 leader_election.cc:302] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e525a82a1e6454081a9fa5c5351994c, 917189f51cfb42fda84a369652aec8da; no voters:
I20260807 19:37:54.318419 19192 raft_consensus.cc:2781] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:37:54.318575 19192 raft_consensus.cc:482] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:37:54.318732 19192 raft_consensus.cc:3037] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:54.321010 19192 raft_consensus.cc:504] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917189f51cfb42fda84a369652aec8da" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45473 } } peers { permanent_uuid: "5e525a82a1e6454081a9fa5c5351994c" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38515 } } peers { permanent_uuid: "d2aff676a6a54a518d6ed8b191a13e93" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44763 } }
I20260807 19:37:54.321631 19192 leader_election.cc:288] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [CANDIDATE]: Term 1 election: Requested vote from peers 917189f51cfb42fda84a369652aec8da (127.14.16.66:45473), d2aff676a6a54a518d6ed8b191a13e93 (127.14.16.67:44763)
I20260807 19:37:54.322074 18983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccb96637d1f44158af1bb79620056512" candidate_uuid: "5e525a82a1e6454081a9fa5c5351994c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917189f51cfb42fda84a369652aec8da"
I20260807 19:37:54.322147 19121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccb96637d1f44158af1bb79620056512" candidate_uuid: "5e525a82a1e6454081a9fa5c5351994c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2aff676a6a54a518d6ed8b191a13e93"
I20260807 19:37:54.322301 18983 raft_consensus.cc:3037] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:54.322321 19121 raft_consensus.cc:3037] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93 [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:37:54.325428 18893 tablet.cc:2369] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:37:54.325598 18983 raft_consensus.cc:2454] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e525a82a1e6454081a9fa5c5351994c in term 1.
I20260807 19:37:54.325598 19121 raft_consensus.cc:2454] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e525a82a1e6454081a9fa5c5351994c in term 1.
I20260807 19:37:54.325989 18774 leader_election.cc:302] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e525a82a1e6454081a9fa5c5351994c, 917189f51cfb42fda84a369652aec8da; no voters:
I20260807 19:37:54.326345 19192 raft_consensus.cc:2781] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:54.326793 19192 raft_consensus.cc:686] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 1 LEADER]: Becoming Leader. State: Replica: 5e525a82a1e6454081a9fa5c5351994c, State: Running, Role: LEADER
I20260807 19:37:54.327350 19192 consensus_queue.cc:237] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917189f51cfb42fda84a369652aec8da" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45473 } } peers { permanent_uuid: "5e525a82a1e6454081a9fa5c5351994c" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38515 } } peers { permanent_uuid: "d2aff676a6a54a518d6ed8b191a13e93" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44763 } }
I20260807 19:37:54.332131 18694 catalog_manager.cc:5495] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e525a82a1e6454081a9fa5c5351994c (127.14.16.65). New cstate: current_term: 1 leader_uuid: "5e525a82a1e6454081a9fa5c5351994c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "917189f51cfb42fda84a369652aec8da" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e525a82a1e6454081a9fa5c5351994c" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2aff676a6a54a518d6ed8b191a13e93" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44763 } health_report { overall_health: UNKNOWN } } }
W20260807 19:37:54.353881 19030 tablet.cc:2369] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:37:54.356935 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.351186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32848 (local address 127.14.16.65:38515)
0807 19:37:54.351498 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:54.351503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:54.351522 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:37:54.351751 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:54.351755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:54.351845 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:54.351993 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:54.352355 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.353066 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.354860 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.355250 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.355438 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.355897 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:54.356365 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:54.356517 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:54.356597 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 19:37:54.356678 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
W20260807 19:37:54.361755 19167 tablet.cc:2369] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:37:54.364666 19121 raft_consensus.cc:1261] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93 [term 1 FOLLOWER]: Refusing update from remote peer 5e525a82a1e6454081a9fa5c5351994c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:54.364717 18983 raft_consensus.cc:1261] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da [term 1 FOLLOWER]: Refusing update from remote peer 5e525a82a1e6454081a9fa5c5351994c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:54.365628 19195 consensus_queue.cc:1035] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2aff676a6a54a518d6ed8b191a13e93" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:54.365994 19192 consensus_queue.cc:1035] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [LEADER]: Connected to new peer: Peer: permanent_uuid: "917189f51cfb42fda84a369652aec8da" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:54.379261 19199 mvcc.cc:205] Tried to move back new op lower bound from 7315994518988390400 to 7315994518848622592. Current Snapshot: MvccSnapshot[applied={T|T < 7315994518988390400}]
I20260807 19:37:54.379594 19198 mvcc.cc:205] Tried to move back new op lower bound from 7315994518988390400 to 7315994518848622592. Current Snapshot: MvccSnapshot[applied={T|T < 7315994518988390400 or (T in {7315994518988390400})}]
I20260807 19:37:54.380731 19200 mvcc.cc:205] Tried to move back new op lower bound from 7315994518988390400 to 7315994518848622592. Current Snapshot: MvccSnapshot[applied={T|T < 7315994518988390400}]
I20260807 19:37:54.390367 18694 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:54.390614 18694 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:54.393949 18694 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9915e1536c964f0ba79b33aac8a9bda9: Sending DeleteTablet for 3 replicas of tablet ccb96637d1f44158af1bb79620056512
I20260807 19:37:54.395066 18962 tablet_service.cc:1514] Processing DeleteTablet for tablet ccb96637d1f44158af1bb79620056512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:54 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:58846
I20260807 19:37:54.395658 18822 tablet_service.cc:1514] Processing DeleteTablet for tablet ccb96637d1f44158af1bb79620056512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:54 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:32846
I20260807 19:37:54.395740 19101 tablet_service.cc:1514] Processing DeleteTablet for tablet ccb96637d1f44158af1bb79620056512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:54 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:37548
I20260807 19:37:54.397094 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 18759
I20260807 19:37:54.407168 19209 tablet_replica.cc:330] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93: stopping tablet replica
I20260807 19:37:54.407733 19209 raft_consensus.cc:2229] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:54.408375 19209 raft_consensus.cc:2258] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:54.411067 19207 tablet_replica.cc:330] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da: stopping tablet replica
I20260807 19:37:54.411566 19207 raft_consensus.cc:2229] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:54.412055 19207 raft_consensus.cc:2258] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:54.412668 19209 ts_tablet_manager.cc:1905] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:54.413589 19208 tablet_replica.cc:330] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c: stopping tablet replica
I20260807 19:37:54.413954 19208 raft_consensus.cc:2229] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:37:54.414165 19207 ts_tablet_manager.cc:1905] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:54.415572 19208 raft_consensus.cc:2258] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 07 19:37:54 dist-test-slave-07c2 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:37:54.420610 19209 ts_tablet_manager.cc:1918] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:37:54.420794 19209 log.cc:1196] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/ccb96637d1f44158af1bb79620056512
I20260807 19:37:54.421188 19208 ts_tablet_manager.cc:1905] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:54.421608 19209 ts_tablet_manager.cc:1939] T ccb96637d1f44158af1bb79620056512 P d2aff676a6a54a518d6ed8b191a13e93: Deleting consensus metadata
I20260807 19:37:54.423605 18678 catalog_manager.cc:4841] TS d2aff676a6a54a518d6ed8b191a13e93 (127.14.16.67:44763): tablet ccb96637d1f44158af1bb79620056512 (table test-table [id=4d57a41fd0ee4b7f96d88902aa4b4aec]) successfully deleted
I20260807 19:37:54.425791 19207 ts_tablet_manager.cc:1918] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:37:54.425964 19207 log.cc:1196] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/ccb96637d1f44158af1bb79620056512
I20260807 19:37:54.426654 19207 ts_tablet_manager.cc:1939] T ccb96637d1f44158af1bb79620056512 P 917189f51cfb42fda84a369652aec8da: Deleting consensus metadata
I20260807 19:37:54.427330 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.398392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.14.16.65:38515)
0807 19:37:54.398581 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:54.398587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:54.404121 (+ 5534us) server_negotiation.cc:407] Connection header received
0807 19:37:54.404333 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:54.404344 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:54.404496 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:54.404608 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:54.408599 (+ 3991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.409883 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.411833 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.413081 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.413222 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.418819 (+ 5597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:54.418833 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:54.418850 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:54.418919 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:54.423121 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:54.424209 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:54.424221 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:54.424224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:54.424446 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:54.424919 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:54.424924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:54.424928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:54.425340 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:54.425678 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:54.426533 (+ 855us) server_negotiation.cc:299] Negotiation successful
0807 19:37:54.426874 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:37:54.428583 18678 catalog_manager.cc:4841] TS 917189f51cfb42fda84a369652aec8da (127.14.16.66:45473): tablet ccb96637d1f44158af1bb79620056512 (table test-table [id=4d57a41fd0ee4b7f96d88902aa4b4aec]) successfully deleted
I20260807 19:37:54.429025 18887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:54.534971 19208 ts_tablet_manager.cc:1918] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:37:54.535264 19208 log.cc:1196] T ccb96637d1f44158af1bb79620056512 P 5e525a82a1e6454081a9fa5c5351994c: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/ccb96637d1f44158af1bb79620056512
I20260807 19:37:54.535760 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 18759
I20260807 19:37:54.556929 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 18898
W20260807 19:37:54.556928 18678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.16.65:38515 (error 108)
I20260807 19:37:54.558838 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.558351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:38515 (local address 127.0.0.1:32860)
0807 19:37:54.558552 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.558681 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:38515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 19:37:54.559057 18678 catalog_manager.cc:4584] TS 5e525a82a1e6454081a9fa5c5351994c (127.14.16.65:38515): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ccb96637d1f44158af1bb79620056512: Network error: Client connection negotiation failed: client connection to 127.14.16.65:38515: connect: Connection refused (error 111)
Aug 07 19:37:54 dist-test-slave-07c2 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:37:54.572672 19179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.557729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.14.16.66:45473)
0807 19:37:54.557944 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:54.557951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:54.558094 (+ 143us) server_negotiation.cc:407] Connection header received
0807 19:37:54.558304 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:54.558316 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:54.558447 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:54.558552 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:54.559979 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.560707 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.561976 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.562346 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.562476 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.566993 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:54.567007 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:54.567023 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:54.567061 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:54.569832 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:54.570521 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:54.570528 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:54.570530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:54.570654 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:54.571047 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:54.571057 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:54.571059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:54.571489 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:54.571807 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:54.572118 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 19:37:54.572354 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 19:37:54.573361 19024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:54.612751 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.612153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:38515 (local address 127.0.0.1:32872)
0807 19:37:54.612447 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.612561 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:38515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 19:37:54.613849 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.613435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:38515 (local address 127.0.0.1:32876)
0807 19:37:54.613599 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.613709 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:38515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 19:37:54.665684 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 18898
I20260807 19:37:54.674572 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.673935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:38515 (local address 127.0.0.1:32886)
0807 19:37:54.674201 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.674338 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:38515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 19:37:54.675804 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.675363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:38515 (local address 127.0.0.1:32902)
0807 19:37:54.675539 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.675642 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:38515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 19:37:54.685617 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 19035
Aug 07 19:37:54 dist-test-slave-07c2 krb5kdc[18650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:37:54.700531 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.686363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37558 (local address 127.14.16.67:44763)
0807 19:37:54.686561 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:54.686567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:54.686586 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:37:54.686782 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:54.686786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:54.686851 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:54.686959 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:54.688336 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.689092 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.690355 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:54.690751 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:54.690896 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:54.695531 (+ 4635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:54.695547 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:54.695564 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:54.695606 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:54.698146 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:54.698746 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:54.698754 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:54.698756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:54.698894 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:54.699221 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:54.699224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:54.699226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:54.699626 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:54.699859 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:54.700111 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 19:37:54.700276 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:37:54.701195 19161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:54.756865 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.756373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:38515 (local address 127.0.0.1:32914)
0807 19:37:54.756608 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.756699 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:38515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 19:37:54.757951 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:54.757559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:38515 (local address 127.0.0.1:32926)
0807 19:37:54.757759 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:54.757840 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:38515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 19:37:54.790211 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 19035
I20260807 19:37:54.809561 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 18663
I20260807 19:37:54.810762 18725 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:54.921378 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 18663
2026-08-07T19:37:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4221 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T19:37:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:54Z Disabled control of system clock
I20260807 19:37:54.997195 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:36347
--webserver_interface=127.14.16.126
--webserver_port=0
--builtin_ntp_servers=127.14.16.84:40473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:36347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 19:37:55.324101 19221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:55.324424 19221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:55.324509 19221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:55.331837 19221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:55.331957 19221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:55.331995 19221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:55.332059 19221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:55.332103 19221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:55.341817 19221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:40473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:36347
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:36347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:55.343099 19221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:55.344900 19221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:55.356909 19228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:55.357004 19227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:55.357004 19230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:55.358155 19221 server_base.cc:1034] running on GCE node
I20260807 19:37:55.359004 19221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:55.360422 19221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:55.361662 19221 hybrid_clock.cc:648] HybridClock initialized: now 1786131475361627 us; error 59 us; skew 500 ppm
I20260807 19:37:55.362006 19221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:37:55.364383 19221 webserver.cc:460] Webserver started at http://127.14.16.126:39603/ using document root <none> and password file <none>
I20260807 19:37:55.365087 19221 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:55.365219 19221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:55.365556 19221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:55.368425 19221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "9f7b72ff1bf74977aff2c8e5f43dc361"
format_stamp: "Formatted at 2026-08-07 19:37:55 on dist-test-slave-07c2"
server_key: "310dd65918bce6689fce4bfd9860cc6d"
server_key_iv: "d5c2e85bb6a3a78175919f7c6b289e29"
server_key_version: "encryptionkey@0"
I20260807 19:37:55.369212 19221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "9f7b72ff1bf74977aff2c8e5f43dc361"
format_stamp: "Formatted at 2026-08-07 19:37:55 on dist-test-slave-07c2"
server_key: "310dd65918bce6689fce4bfd9860cc6d"
server_key_iv: "d5c2e85bb6a3a78175919f7c6b289e29"
server_key_version: "encryptionkey@0"
I20260807 19:37:55.374110 19221 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 19:37:55.377556 19236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:55.378835 19221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:37:55.379010 19221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "9f7b72ff1bf74977aff2c8e5f43dc361"
format_stamp: "Formatted at 2026-08-07 19:37:55 on dist-test-slave-07c2"
server_key: "310dd65918bce6689fce4bfd9860cc6d"
server_key_iv: "d5c2e85bb6a3a78175919f7c6b289e29"
server_key_version: "encryptionkey@0"
I20260807 19:37:55.379221 19221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:55.384578 19221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:55.385308 19221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:55.385599 19221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:55.403825 19221 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:36347
I20260807 19:37:55.403870 19287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:36347 every 8 connection(s)
I20260807 19:37:55.405248 19221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:55.407819 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 19221
I20260807 19:37:55.408021 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:55.408892 14401 external_mini_cluster.cc:1428] Setting key 1b27fc733296cc42b5e461d7b24ae647
I20260807 19:37:55.409757 19288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:55.417662 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:55.411010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45514 (local address 127.14.16.126:36347)
0807 19:37:55.412047 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:55.412058 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:55.412097 (+ 39us) server_negotiation.cc:407] Connection header received
0807 19:37:55.414227 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:55.414248 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:55.414643 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:55.414964 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:55.416112 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:55.416118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:55.416130 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:55.416144 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:55.416211 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:55.416229 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:55.416412 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:55.416513 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 19:37:55.416781 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":430,"threads_started":1}
I20260807 19:37:55.422821 19288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361: Bootstrap starting.
I20260807 19:37:55.425669 19288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:55.426810 19288 log.cc:824] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:55.429359 19288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361: No bootstrap required, opened a new log
I20260807 19:37:55.436450 19288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f7b72ff1bf74977aff2c8e5f43dc361" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36347 } }
I20260807 19:37:55.436800 19288 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:55.436919 19288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f7b72ff1bf74977aff2c8e5f43dc361, State: Initialized, Role: FOLLOWER
I20260807 19:37:55.437451 19288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f7b72ff1bf74977aff2c8e5f43dc361" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36347 } }
I20260807 19:37:55.437685 19288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:55.437819 19288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:55.437994 19288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:55.440143 19288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f7b72ff1bf74977aff2c8e5f43dc361" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36347 } }
I20260807 19:37:55.440644 19288 leader_election.cc:302] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f7b72ff1bf74977aff2c8e5f43dc361; no voters:
I20260807 19:37:55.441350 19288 leader_election.cc:288] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:55.441465 19294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:55.442201 19294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [term 1 LEADER]: Becoming Leader. State: Replica: 9f7b72ff1bf74977aff2c8e5f43dc361, State: Running, Role: LEADER
I20260807 19:37:55.442814 19294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f7b72ff1bf74977aff2c8e5f43dc361" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36347 } }
I20260807 19:37:55.443660 19288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:55.445747 19295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f7b72ff1bf74977aff2c8e5f43dc361" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f7b72ff1bf74977aff2c8e5f43dc361" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36347 } } }
I20260807 19:37:55.446107 19295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:55.447675 19296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f7b72ff1bf74977aff2c8e5f43dc361. Latest consensus state: current_term: 1 leader_uuid: "9f7b72ff1bf74977aff2c8e5f43dc361" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f7b72ff1bf74977aff2c8e5f43dc361" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36347 } } }
I20260807 19:37:55.448164 19296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:55.448964 19300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:55.454651 19300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:55.463371 19300 catalog_manager.cc:1347] Generated new cluster ID: 5946538838c8499ea64509be820df832
I20260807 19:37:55.463507 19300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:55.475769 19300 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:55.476246 19300 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:55.490800 19300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361: Generated new TSK 0
I20260807 19:37:55.491921 19300 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:37:55.504243 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:36347
--builtin_ntp_servers=127.14.16.84:40473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 19:37:55.840111 19313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:55.840417 19313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:55.840584 19313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:55.847792 19313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:55.847916 19313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:55.848093 19313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:37:55.857017 19313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:40473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:36347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:55.858240 19313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:55.859957 19313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:55.872366 19319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:55.872566 19318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:55.872542 19321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:55.873411 19313 server_base.cc:1034] running on GCE node
I20260807 19:37:55.874035 19313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:55.874907 19313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:55.876129 19313 hybrid_clock.cc:648] HybridClock initialized: now 1786131475876081 us; error 36 us; skew 500 ppm
I20260807 19:37:55.876475 19313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:37:55.878664 19313 webserver.cc:460] Webserver started at http://127.14.16.65:40549/ using document root <none> and password file <none>
I20260807 19:37:55.879415 19313 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:55.879542 19313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:55.879889 19313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:55.882673 19313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "7940b66efe8347538288dd93f443c83f"
format_stamp: "Formatted at 2026-08-07 19:37:55 on dist-test-slave-07c2"
server_key: "66fc87eabfa664e312fb623998923daf"
server_key_iv: "1295825b4dcb368105e1a2f34a1db5f7"
server_key_version: "encryptionkey@0"
I20260807 19:37:55.883567 19313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "7940b66efe8347538288dd93f443c83f"
format_stamp: "Formatted at 2026-08-07 19:37:55 on dist-test-slave-07c2"
server_key: "66fc87eabfa664e312fb623998923daf"
server_key_iv: "1295825b4dcb368105e1a2f34a1db5f7"
server_key_version: "encryptionkey@0"
I20260807 19:37:55.888737 19313 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:37:55.892218 19327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:55.893570 19313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:37:55.893841 19313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "7940b66efe8347538288dd93f443c83f"
format_stamp: "Formatted at 2026-08-07 19:37:55 on dist-test-slave-07c2"
server_key: "66fc87eabfa664e312fb623998923daf"
server_key_iv: "1295825b4dcb368105e1a2f34a1db5f7"
server_key_version: "encryptionkey@0"
I20260807 19:37:55.894084 19313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:55.899864 19313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:55.900615 19313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:55.900887 19313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:55.901782 19313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:55.903340 19313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:55.903470 19313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:55.903597 19313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:55.903694 19313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:55.936959 19313 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:45485
I20260807 19:37:55.937014 19442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:45485 every 8 connection(s)
I20260807 19:37:55.938323 19313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:37:55.947057 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:55.940865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:53339 (local address 127.14.16.126:36347)
0807 19:37:55.941359 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:55.941366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:55.942488 (+ 1122us) server_negotiation.cc:407] Connection header received
0807 19:37:55.944736 (+ 2248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:55.944744 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:55.944820 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:55.944962 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:55.946280 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:55.946286 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:55.946290 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:55.946310 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:55.946358 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:55.946371 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:55.946497 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:55.946694 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 19:37:55.946773 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":236,"threads_started":1}
I20260807 19:37:55.947829 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 19313
I20260807 19:37:55.948032 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:37:55.948251 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:55.941350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:36347 (local address 127.14.16.65:53339)
0807 19:37:55.942337 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:55.942373 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:55.944366 (+ 1993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:55.945177 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:55.945188 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:55.945290 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:55.945986 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:37:55.945988 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:55.946024 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:37:55.946106 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:55.946467 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:55.946478 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:55.946485 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 19:37:55.946761 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 19:37:55.947002 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":351,"threads_started":1}
I20260807 19:37:55.948915 14401 external_mini_cluster.cc:1428] Setting key 4cd6adc0958c4ec938d14813b2b81785
I20260807 19:37:55.949994 19443 heartbeater.cc:347] Connected to a master server at 127.14.16.126:36347
I20260807 19:37:55.950343 19443 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:55.951041 19443 heartbeater.cc:511] Master 127.14.16.126:36347 requested a full tablet report, sending...
I20260807 19:37:55.951946 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:36347
--builtin_ntp_servers=127.14.16.84:40473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 19:37:55.953372 19253 ts_manager.cc:194] Registered new tserver with Master: 7940b66efe8347538288dd93f443c83f (127.14.16.65:45485)
W20260807 19:37:56.277606 19447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:56.278057 19447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:56.278239 19447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:56.286145 19447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:56.286257 19447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:56.286377 19447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:37:56.295919 19447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:40473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:36347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:56.297173 19447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:56.299065 19447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:56.312307 19452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:56.312383 19455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:56.312345 19453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:56.313006 19447 server_base.cc:1034] running on GCE node
I20260807 19:37:56.313689 19447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:56.314630 19447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:56.315855 19447 hybrid_clock.cc:648] HybridClock initialized: now 1786131476315815 us; error 54 us; skew 500 ppm
I20260807 19:37:56.316181 19447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:37:56.318506 19447 webserver.cc:460] Webserver started at http://127.14.16.66:33523/ using document root <none> and password file <none>
I20260807 19:37:56.319196 19447 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:56.319358 19447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:56.319697 19447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:56.322624 19447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "6e642e1c42524c80a59d23fd64d38662"
format_stamp: "Formatted at 2026-08-07 19:37:56 on dist-test-slave-07c2"
server_key: "5e2ff4a2c8007a066dc80292c53f8211"
server_key_iv: "6b506595a19b741b6eae99e348425c15"
server_key_version: "encryptionkey@0"
I20260807 19:37:56.323493 19447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "6e642e1c42524c80a59d23fd64d38662"
format_stamp: "Formatted at 2026-08-07 19:37:56 on dist-test-slave-07c2"
server_key: "5e2ff4a2c8007a066dc80292c53f8211"
server_key_iv: "6b506595a19b741b6eae99e348425c15"
server_key_version: "encryptionkey@0"
I20260807 19:37:56.328498 19447 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:37:56.331882 19461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:56.333271 19447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:37:56.333436 19447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "6e642e1c42524c80a59d23fd64d38662"
format_stamp: "Formatted at 2026-08-07 19:37:56 on dist-test-slave-07c2"
server_key: "5e2ff4a2c8007a066dc80292c53f8211"
server_key_iv: "6b506595a19b741b6eae99e348425c15"
server_key_version: "encryptionkey@0"
I20260807 19:37:56.333655 19447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:56.339087 19447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:56.339908 19447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:56.340181 19447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:56.341061 19447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:56.342422 19447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:56.342553 19447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:56.342674 19447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:56.342758 19447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:56.375726 19447 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:33465
I20260807 19:37:56.375815 19576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:33465 every 8 connection(s)
I20260807 19:37:56.377038 19447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:37:56.383826 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 19447
I20260807 19:37:56.384019 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:37:56.384471 14401 external_mini_cluster.cc:1428] Setting key 7405de88e22a502c47e228b8ef15a83b
I20260807 19:37:56.385115 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.379569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:52139 (local address 127.14.16.126:36347)
0807 19:37:56.379777 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:56.379781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:56.381114 (+ 1333us) server_negotiation.cc:407] Connection header received
0807 19:37:56.382804 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:56.382808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:56.382876 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:56.382963 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:56.384368 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:56.384374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:56.384378 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:56.384402 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:56.384434 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:56.384446 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:56.384568 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:56.384760 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 19:37:56.384831 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:37:56.386170 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.380070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:36347 (local address 127.14.16.66:52139)
0807 19:37:56.380952 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:56.381006 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:56.382495 (+ 1489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:56.383174 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:56.383181 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:56.383264 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:56.384049 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:37:56.384051 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:56.384106 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:37:56.384206 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:56.384539 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:56.384550 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:56.384557 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 19:37:56.384840 (+ 283us) client_negotiation.cc:240] Negotiation successful
0807 19:37:56.385067 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":265,"threads_started":1}
I20260807 19:37:56.387418 19577 heartbeater.cc:347] Connected to a master server at 127.14.16.126:36347
I20260807 19:37:56.387511 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:36347
--builtin_ntp_servers=127.14.16.84:40473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 19:37:56.387737 19577 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:56.388403 19577 heartbeater.cc:511] Master 127.14.16.126:36347 requested a full tablet report, sending...
I20260807 19:37:56.389925 19253 ts_manager.cc:194] Registered new tserver with Master: 6e642e1c42524c80a59d23fd64d38662 (127.14.16.66:33465)
W20260807 19:37:56.723635 19580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:56.723924 19580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:56.724097 19580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:56.731302 19580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:56.731431 19580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:56.731631 19580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:37:56.740752 19580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:40473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:36347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:56.742044 19580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:56.743894 19580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:56.756978 19585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:56.756983 19588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:56.757021 19586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:56.757845 19580 server_base.cc:1034] running on GCE node
I20260807 19:37:56.758387 19580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:56.759244 19580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:56.760495 19580 hybrid_clock.cc:648] HybridClock initialized: now 1786131476760459 us; error 28 us; skew 500 ppm
I20260807 19:37:56.760815 19580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:37:56.762996 19580 webserver.cc:460] Webserver started at http://127.14.16.67:45971/ using document root <none> and password file <none>
I20260807 19:37:56.763736 19580 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:56.763854 19580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:56.764192 19580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:56.767203 19580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "ec5948e302a945b7bcf9648223148afc"
format_stamp: "Formatted at 2026-08-07 19:37:56 on dist-test-slave-07c2"
server_key: "4a435576f1b67a938725980be1ec6120"
server_key_iv: "b95be0848b187d8405bf6349c559e3ba"
server_key_version: "encryptionkey@0"
I20260807 19:37:56.768054 19580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "ec5948e302a945b7bcf9648223148afc"
format_stamp: "Formatted at 2026-08-07 19:37:56 on dist-test-slave-07c2"
server_key: "4a435576f1b67a938725980be1ec6120"
server_key_iv: "b95be0848b187d8405bf6349c559e3ba"
server_key_version: "encryptionkey@0"
I20260807 19:37:56.772960 19580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:37:56.776412 19594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:56.777745 19580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:37:56.777936 19580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "ec5948e302a945b7bcf9648223148afc"
format_stamp: "Formatted at 2026-08-07 19:37:56 on dist-test-slave-07c2"
server_key: "4a435576f1b67a938725980be1ec6120"
server_key_iv: "b95be0848b187d8405bf6349c559e3ba"
server_key_version: "encryptionkey@0"
I20260807 19:37:56.778168 19580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:56.783877 19580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:56.784653 19580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:56.784934 19580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:56.785817 19580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:37:56.787155 19580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:37:56.787328 19580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:56.787462 19580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:37:56.787559 19580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:56.822029 19580 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:40721
I20260807 19:37:56.822085 19707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:40721 every 8 connection(s)
I20260807 19:37:56.823504 19580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:37:56.830170 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 19580
I20260807 19:37:56.830363 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:37:56.830948 14401 external_mini_cluster.cc:1428] Setting key 60697f5cdb9c50b9ad0fb221cbc64b0a
I20260807 19:37:56.831990 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.826132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:59145 (local address 127.14.16.126:36347)
0807 19:37:56.826352 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:56.826357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:56.827706 (+ 1349us) server_negotiation.cc:407] Connection header received
0807 19:37:56.829339 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:56.829343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:56.829395 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:56.829484 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:56.831063 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:56.831103 (+ 40us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:56.831108 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:56.831130 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:56.831176 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:56.831193 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:56.831353 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:56.831602 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 19:37:56.831669 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:37:56.832993 19710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.826608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:36347 (local address 127.14.16.67:59145)
0807 19:37:56.827549 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:56.827585 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:56.829105 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:56.829674 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:56.829681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:56.829835 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:56.830674 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:37:56.830678 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:56.830727 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:37:56.830844 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:56.831347 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:56.831358 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:56.831367 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 19:37:56.831640 (+ 273us) client_negotiation.cc:240] Negotiation successful
0807 19:37:56.831867 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":260,"threads_started":1}
I20260807 19:37:56.834383 19708 heartbeater.cc:347] Connected to a master server at 127.14.16.126:36347
I20260807 19:37:56.834664 19708 heartbeater.cc:464] Registering TS with master...
I20260807 19:37:56.835243 19708 heartbeater.cc:511] Master 127.14.16.126:36347 requested a full tablet report, sending...
I20260807 19:37:56.836584 19253 ts_manager.cc:194] Registered new tserver with Master: ec5948e302a945b7bcf9648223148afc (127.14.16.67:40721)
I20260807 19:37:56.846235 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:37:56.853281 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.850651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45526 (local address 127.14.16.126:36347)
0807 19:37:56.850834 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:56.850842 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:56.851385 (+ 543us) server_negotiation.cc:407] Connection header received
0807 19:37:56.851599 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:56.851610 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:56.851660 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:56.851762 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:56.852627 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:56.852633 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:56.852637 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:56.852662 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:56.852691 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:56.852702 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:56.852823 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:56.852932 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 19:37:56.852998 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:37:56.856964 19253 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:37:56.863708 19253 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:37:56.881970 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.876379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.14.16.65:45485)
0807 19:37:56.877035 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:56.877047 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:56.878066 (+ 1019us) server_negotiation.cc:407] Connection header received
0807 19:37:56.878155 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:56.878164 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:56.878397 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:56.878556 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:56.880318 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:56.880328 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:56.880345 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:56.880372 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:56.880429 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:56.880452 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:56.880584 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:56.881526 (+ 942us) server_negotiation.cc:299] Negotiation successful
0807 19:37:56.881631 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":305,"threads_started":1}
I20260807 19:37:56.882424 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.875694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:45485 (local address 127.0.0.1:46720)
0807 19:37:56.877441 (+ 1747us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:56.877464 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:56.877599 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:56.878596 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:56.878605 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:56.878686 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:56.879756 (+ 1070us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:37:56.879760 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:56.880088 (+ 328us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:37:56.880136 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:56.880635 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:56.880649 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:56.880659 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 19:37:56.880847 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 19:37:56.880935 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1551,"thread_start_us":1350,"threads_started":1}
I20260807 19:37:56.882689 19721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.875249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:33465 (local address 127.0.0.1:53636)
0807 19:37:56.876336 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:56.876376 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:56.876494 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:56.879193 (+ 2699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:56.879199 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:56.879224 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:56.879874 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:37:56.879877 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:56.879896 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:37:56.879981 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:56.881973 (+ 1992us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:56.881978 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:56.881983 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:37:56.882118 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 19:37:56.882174 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"thread_start_us":725,"threads_started":1}
I20260807 19:37:56.882692 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.875451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.14.16.66:33465)
0807 19:37:56.878111 (+ 2660us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:56.878118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:56.878139 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:37:56.878198 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:56.878204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:56.878396 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:56.880746 (+ 2350us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:56.881741 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:56.881749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:56.881761 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:56.881778 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:56.881826 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:56.881850 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:56.882032 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:56.882103 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 19:37:56.882209 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2500,"thread_start_us":2381,"threads_started":1}
I20260807 19:37:56.885077 19726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.877773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.14.16.67:40721)
0807 19:37:56.881778 (+ 4005us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:56.881784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:56.881812 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:37:56.881872 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:56.881877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:56.882091 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:56.882231 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:56.884202 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:56.884211 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:56.884224 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:56.884245 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:56.884301 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:56.884328 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:56.884520 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:56.884585 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 19:37:56.884667 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3852,"thread_start_us":3673,"threads_started":1}
I20260807 19:37:56.885076 19725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.877440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:40721 (local address 127.0.0.1:34766)
0807 19:37:56.880468 (+ 3028us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:56.880486 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:56.880570 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:56.882428 (+ 1858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:56.882432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:56.882458 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:56.883965 (+ 1507us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:37:56.883967 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:56.883983 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:37:56.884027 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:56.884435 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:56.884440 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:56.884445 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:37:56.884571 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 19:37:56.884622 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2909,"thread_start_us":2764,"threads_started":1}
I20260807 19:37:56.886557 19378 tablet_service.cc:1467] Processing CreateTablet for tablet e2c62bece6624e328d8e74a7ba4e7106 (DEFAULT_TABLE table=test-table [id=05c0ea53af394be28c76fb2505c425cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:56.886557 19510 tablet_service.cc:1467] Processing CreateTablet for tablet e2c62bece6624e328d8e74a7ba4e7106 (DEFAULT_TABLE table=test-table [id=05c0ea53af394be28c76fb2505c425cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:56.888022 19510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2c62bece6624e328d8e74a7ba4e7106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:56.888023 19378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2c62bece6624e328d8e74a7ba4e7106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:56.888423 19642 tablet_service.cc:1467] Processing CreateTablet for tablet e2c62bece6624e328d8e74a7ba4e7106 (DEFAULT_TABLE table=test-table [id=05c0ea53af394be28c76fb2505c425cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:37:56.889493 19642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2c62bece6624e328d8e74a7ba4e7106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:56.900451 19727 tablet_bootstrap.cc:492] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc: Bootstrap starting.
I20260807 19:37:56.903610 19728 tablet_bootstrap.cc:492] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662: Bootstrap starting.
I20260807 19:37:56.903693 19729 tablet_bootstrap.cc:492] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f: Bootstrap starting.
I20260807 19:37:56.905100 19727 tablet_bootstrap.cc:654] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:56.906368 19727 log.cc:824] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:56.906447 19729 tablet_bootstrap.cc:654] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:56.906950 19728 tablet_bootstrap.cc:654] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:56.907660 19729 log.cc:824] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:56.908102 19728 log.cc:824] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:56.909921 19727 tablet_bootstrap.cc:492] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc: No bootstrap required, opened a new log
I20260807 19:37:56.910265 19727 ts_tablet_manager.cc:1397] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 19:37:56.910677 19729 tablet_bootstrap.cc:492] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f: No bootstrap required, opened a new log
I20260807 19:37:56.910938 19729 ts_tablet_manager.cc:1397] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 19:37:56.910975 19728 tablet_bootstrap.cc:492] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662: No bootstrap required, opened a new log
I20260807 19:37:56.911221 19728 ts_tablet_manager.cc:1397] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260807 19:37:56.918018 19728 raft_consensus.cc:348] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e642e1c42524c80a59d23fd64d38662" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33465 } } peers { permanent_uuid: "7940b66efe8347538288dd93f443c83f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 45485 } } peers { permanent_uuid: "ec5948e302a945b7bcf9648223148afc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40721 } }
I20260807 19:37:56.918485 19728 raft_consensus.cc:374] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:56.918581 19728 raft_consensus.cc:729] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e642e1c42524c80a59d23fd64d38662, State: Initialized, Role: FOLLOWER
I20260807 19:37:56.919450 19728 consensus_queue.cc:260] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e642e1c42524c80a59d23fd64d38662" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33465 } } peers { permanent_uuid: "7940b66efe8347538288dd93f443c83f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 45485 } } peers { permanent_uuid: "ec5948e302a945b7bcf9648223148afc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40721 } }
I20260807 19:37:56.920696 19577 heartbeater.cc:503] Master 127.14.16.126:36347 was elected leader, sending a full tablet report...
I20260807 19:37:56.920938 19727 raft_consensus.cc:348] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e642e1c42524c80a59d23fd64d38662" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33465 } } peers { permanent_uuid: "7940b66efe8347538288dd93f443c83f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 45485 } } peers { permanent_uuid: "ec5948e302a945b7bcf9648223148afc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40721 } }
I20260807 19:37:56.921350 19727 raft_consensus.cc:374] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:56.921361 19728 ts_tablet_manager.cc:1428] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260807 19:37:56.921455 19727 raft_consensus.cc:729] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5948e302a945b7bcf9648223148afc, State: Initialized, Role: FOLLOWER
I20260807 19:37:56.921454 19729 raft_consensus.cc:348] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e642e1c42524c80a59d23fd64d38662" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33465 } } peers { permanent_uuid: "7940b66efe8347538288dd93f443c83f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 45485 } } peers { permanent_uuid: "ec5948e302a945b7bcf9648223148afc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40721 } }
I20260807 19:37:56.921867 19729 raft_consensus.cc:374] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:56.921969 19729 raft_consensus.cc:729] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7940b66efe8347538288dd93f443c83f, State: Initialized, Role: FOLLOWER
I20260807 19:37:56.922156 19727 consensus_queue.cc:260] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e642e1c42524c80a59d23fd64d38662" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33465 } } peers { permanent_uuid: "7940b66efe8347538288dd93f443c83f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 45485 } } peers { permanent_uuid: "ec5948e302a945b7bcf9648223148afc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40721 } }
I20260807 19:37:56.922530 19729 consensus_queue.cc:260] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e642e1c42524c80a59d23fd64d38662" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33465 } } peers { permanent_uuid: "7940b66efe8347538288dd93f443c83f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 45485 } } peers { permanent_uuid: "ec5948e302a945b7bcf9648223148afc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40721 } }
I20260807 19:37:56.923240 19708 heartbeater.cc:503] Master 127.14.16.126:36347 was elected leader, sending a full tablet report...
I20260807 19:37:56.924109 19727 ts_tablet_manager.cc:1428] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 19:37:56.925034 19443 heartbeater.cc:503] Master 127.14.16.126:36347 was elected leader, sending a full tablet report...
I20260807 19:37:56.925599 19729 ts_tablet_manager.cc:1428] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260807 19:37:56.933815 19735 raft_consensus.cc:482] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:37:56.934019 19735 raft_consensus.cc:504] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e642e1c42524c80a59d23fd64d38662" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33465 } } peers { permanent_uuid: "7940b66efe8347538288dd93f443c83f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 45485 } } peers { permanent_uuid: "ec5948e302a945b7bcf9648223148afc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40721 } }
I20260807 19:37:56.935418 19735 leader_election.cc:288] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e642e1c42524c80a59d23fd64d38662 (127.14.16.66:33465), ec5948e302a945b7bcf9648223148afc (127.14.16.67:40721)
I20260807 19:37:56.939242 19726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.935800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:37787 (local address 127.14.16.67:40721)
0807 19:37:56.935951 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:56.935957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:56.936752 (+ 795us) server_negotiation.cc:407] Connection header received
0807 19:37:56.936955 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:56.936960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:56.937016 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:56.937086 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:56.938656 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:56.938661 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:56.938666 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:56.938685 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:56.938736 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:56.938752 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:56.938891 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:56.938942 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 19:37:56.939003 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:37:56.939509 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.935586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:40721 (local address 127.14.16.65:37787)
0807 19:37:56.936612 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:56.936636 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:56.936785 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:56.937286 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:56.937291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:56.937314 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:56.938410 (+ 1096us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:37:56.938415 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:56.938439 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:37:56.938493 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:56.938859 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:56.938862 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:56.938865 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 19:37:56.938980 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 19:37:56.939024 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":1171,"threads_started":1}
I20260807 19:37:56.940186 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.935764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:58867 (local address 127.14.16.66:33465)
0807 19:37:56.935940 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:56.935947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:56.936747 (+ 800us) server_negotiation.cc:407] Connection header received
0807 19:37:56.936980 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:56.936984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:56.937031 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:56.937097 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:56.939464 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:56.939470 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:56.939475 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:56.939494 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:56.939530 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:56.939544 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:56.939656 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:56.939863 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 19:37:56.939930 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:37:56.940362 19736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.935587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:33465 (local address 127.14.16.65:58867)
0807 19:37:56.936649 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:56.936675 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:56.936809 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:56.937286 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:56.937291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:56.937314 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:56.938467 (+ 1153us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:37:56.938470 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:56.938484 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:37:56.938516 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:56.939707 (+ 1191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:56.939712 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:56.939717 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:37:56.939861 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 19:37:56.939910 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":78,"thread_start_us":483,"threads_started":1}
I20260807 19:37:56.940490 19663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2c62bece6624e328d8e74a7ba4e7106" candidate_uuid: "7940b66efe8347538288dd93f443c83f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5948e302a945b7bcf9648223148afc" is_pre_election: true
I20260807 19:37:56.940982 19663 raft_consensus.cc:2454] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7940b66efe8347538288dd93f443c83f in term 0.
I20260807 19:37:56.941200 19530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2c62bece6624e328d8e74a7ba4e7106" candidate_uuid: "7940b66efe8347538288dd93f443c83f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e642e1c42524c80a59d23fd64d38662" is_pre_election: true
W20260807 19:37:56.941591 19444 tablet.cc:2369] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:37:56.941699 19530 raft_consensus.cc:2454] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7940b66efe8347538288dd93f443c83f in term 0.
I20260807 19:37:56.941691 19331 leader_election.cc:302] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7940b66efe8347538288dd93f443c83f, ec5948e302a945b7bcf9648223148afc; no voters:
I20260807 19:37:56.942121 19735 raft_consensus.cc:2781] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:37:56.942377 19735 raft_consensus.cc:482] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:37:56.942507 19735 raft_consensus.cc:3037] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:56.945078 19735 raft_consensus.cc:504] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e642e1c42524c80a59d23fd64d38662" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33465 } } peers { permanent_uuid: "7940b66efe8347538288dd93f443c83f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 45485 } } peers { permanent_uuid: "ec5948e302a945b7bcf9648223148afc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40721 } }
I20260807 19:37:56.945696 19735 leader_election.cc:288] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [CANDIDATE]: Term 1 election: Requested vote from peers 6e642e1c42524c80a59d23fd64d38662 (127.14.16.66:33465), ec5948e302a945b7bcf9648223148afc (127.14.16.67:40721)
I20260807 19:37:56.946092 19530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2c62bece6624e328d8e74a7ba4e7106" candidate_uuid: "7940b66efe8347538288dd93f443c83f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e642e1c42524c80a59d23fd64d38662"
I20260807 19:37:56.946200 19663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2c62bece6624e328d8e74a7ba4e7106" candidate_uuid: "7940b66efe8347538288dd93f443c83f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5948e302a945b7bcf9648223148afc"
I20260807 19:37:56.946295 19530 raft_consensus.cc:3037] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:56.946399 19663 raft_consensus.cc:3037] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:56.948897 19530 raft_consensus.cc:2454] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7940b66efe8347538288dd93f443c83f in term 1.
I20260807 19:37:56.949307 19328 leader_election.cc:302] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e642e1c42524c80a59d23fd64d38662, 7940b66efe8347538288dd93f443c83f; no voters:
I20260807 19:37:56.949609 19735 raft_consensus.cc:2781] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:56.949683 19663 raft_consensus.cc:2454] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7940b66efe8347538288dd93f443c83f in term 1.
I20260807 19:37:56.950351 19735 raft_consensus.cc:686] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [term 1 LEADER]: Becoming Leader. State: Replica: 7940b66efe8347538288dd93f443c83f, State: Running, Role: LEADER
I20260807 19:37:56.951541 19735 consensus_queue.cc:237] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e642e1c42524c80a59d23fd64d38662" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33465 } } peers { permanent_uuid: "7940b66efe8347538288dd93f443c83f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 45485 } } peers { permanent_uuid: "ec5948e302a945b7bcf9648223148afc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40721 } }
I20260807 19:37:56.956192 19253 catalog_manager.cc:5495] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7940b66efe8347538288dd93f443c83f (127.14.16.65). New cstate: current_term: 1 leader_uuid: "7940b66efe8347538288dd93f443c83f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e642e1c42524c80a59d23fd64d38662" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7940b66efe8347538288dd93f443c83f" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 45485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec5948e302a945b7bcf9648223148afc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260807 19:37:56.987248 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:56.985001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46732 (local address 127.14.16.65:45485)
0807 19:37:56.985194 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:56.985200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:56.985313 (+ 113us) server_negotiation.cc:407] Connection header received
0807 19:37:56.985564 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:56.985568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:56.985646 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:56.985754 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:56.986617 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:56.986623 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:56.986627 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:56.986644 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:56.986684 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:56.986695 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:56.986816 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:56.986911 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 19:37:56.986970 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:37:56.994974 19663 raft_consensus.cc:1261] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc [term 1 FOLLOWER]: Refusing update from remote peer 7940b66efe8347538288dd93f443c83f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:56.994989 19530 raft_consensus.cc:1261] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662 [term 1 FOLLOWER]: Refusing update from remote peer 7940b66efe8347538288dd93f443c83f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:56.996007 19738 consensus_queue.cc:1035] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e642e1c42524c80a59d23fd64d38662" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:56.996364 19735 consensus_queue.cc:1035] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec5948e302a945b7bcf9648223148afc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:57.008937 19741 mvcc.cc:205] Tried to move back new op lower bound from 7315994529762127872 to 7315994529597558784. Current Snapshot: MvccSnapshot[applied={T|T < 7315994529762127872}]
I20260807 19:37:57.022284 19252 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:57.022586 19252 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:37:57.026598 19252 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f7b72ff1bf74977aff2c8e5f43dc361: Sending DeleteTablet for 3 replicas of tablet e2c62bece6624e328d8e74a7ba4e7106
I20260807 19:37:57.027817 19510 tablet_service.cc:1514] Processing DeleteTablet for tablet e2c62bece6624e328d8e74a7ba4e7106 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:57 UTC) from {username='slave'} at 127.0.0.1:53636
I20260807 19:37:57.028167 19642 tablet_service.cc:1514] Processing DeleteTablet for tablet e2c62bece6624e328d8e74a7ba4e7106 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:57 UTC) from {username='slave'} at 127.0.0.1:34766
I20260807 19:37:57.028424 19378 tablet_service.cc:1514] Processing DeleteTablet for tablet e2c62bece6624e328d8e74a7ba4e7106 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:37:57 UTC) from {username='slave'} at 127.0.0.1:46720
I20260807 19:37:57.030445 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 19313
I20260807 19:37:57.040658 19752 tablet_replica.cc:330] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662: stopping tablet replica
I20260807 19:37:57.041309 19752 raft_consensus.cc:2229] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:57.041982 19752 raft_consensus.cc:2258] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:57.046934 19752 ts_tablet_manager.cc:1905] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:57.047868 19753 tablet_replica.cc:330] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc: stopping tablet replica
I20260807 19:37:57.048324 19753 raft_consensus.cc:2229] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:37:57.048867 19753 raft_consensus.cc:2258] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:37:57.050464 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:57.032435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46736 (local address 127.14.16.65:45485)
0807 19:37:57.032678 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:57.032688 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:57.045981 (+ 13293us) server_negotiation.cc:407] Connection header received
0807 19:37:57.046568 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:57.046574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:57.046647 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:57.046724 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:57.049555 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:57.049560 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:57.049565 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:57.049589 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:57.049626 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:57.049641 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:57.049739 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:57.050012 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 19:37:57.050085 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:37:57.051627 19438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:57.055734 19752 ts_tablet_manager.cc:1918] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:37:57.055941 19752 log.cc:1196] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/e2c62bece6624e328d8e74a7ba4e7106
I20260807 19:37:57.056573 19752 ts_tablet_manager.cc:1939] T e2c62bece6624e328d8e74a7ba4e7106 P 6e642e1c42524c80a59d23fd64d38662: Deleting consensus metadata
I20260807 19:37:57.057372 19753 ts_tablet_manager.cc:1905] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:37:57.058535 19237 catalog_manager.cc:4841] TS 6e642e1c42524c80a59d23fd64d38662 (127.14.16.66:33465): tablet e2c62bece6624e328d8e74a7ba4e7106 (table test-table [id=05c0ea53af394be28c76fb2505c425cd]) successfully deleted
I20260807 19:37:57.062407 19753 ts_tablet_manager.cc:1918] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:37:57.062605 19753 log.cc:1196] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/e2c62bece6624e328d8e74a7ba4e7106
I20260807 19:37:57.063063 19753 ts_tablet_manager.cc:1939] T e2c62bece6624e328d8e74a7ba4e7106 P ec5948e302a945b7bcf9648223148afc: Deleting consensus metadata
I20260807 19:37:57.064968 19240 catalog_manager.cc:4841] TS ec5948e302a945b7bcf9648223148afc (127.14.16.67:40721): tablet e2c62bece6624e328d8e74a7ba4e7106 (table test-table [id=05c0ea53af394be28c76fb2505c425cd]) successfully deleted
I20260807 19:37:57.152992 19754 tablet_replica.cc:330] T e2c62bece6624e328d8e74a7ba4e7106 P 7940b66efe8347538288dd93f443c83f: stopping tablet replica
I20260807 19:37:57.153012 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 19313
W20260807 19:37:57.173264 19240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.16.65:45485 (error 108)
I20260807 19:37:57.174031 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 19447
I20260807 19:37:57.175366 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:57.174834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:45485 (local address 127.0.0.1:46742)
0807 19:37:57.175057 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:57.175176 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:45485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 19:37:57.175621 19240 catalog_manager.cc:4584] TS 7940b66efe8347538288dd93f443c83f (127.14.16.65:45485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e2c62bece6624e328d8e74a7ba4e7106: Network error: Client connection negotiation failed: client connection to 127.14.16.65:45485: connect: Connection refused (error 111)
I20260807 19:37:57.177316 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:57.174903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53646 (local address 127.14.16.66:33465)
0807 19:37:57.175083 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:57.175089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:57.175245 (+ 156us) server_negotiation.cc:407] Connection header received
0807 19:37:57.175482 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:57.175487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:57.175605 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:57.175720 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:57.176653 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:57.176659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:57.176664 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:57.176682 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:57.176724 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:57.176736 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:57.176873 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:57.176968 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 19:37:57.177031 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:37:57.177951 19572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:57.229481 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:57.228894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:45485 (local address 127.0.0.1:46758)
0807 19:37:57.229147 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:57.229275 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:45485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 19:37:57.230579 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:57.230177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:45485 (local address 127.0.0.1:46774)
0807 19:37:57.230344 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:57.230429 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:45485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 19:37:57.268254 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 19447
I20260807 19:37:57.287467 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 19580
I20260807 19:37:57.290199 19726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:57.288219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34778 (local address 127.14.16.67:40721)
0807 19:37:57.288391 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:57.288396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:57.288428 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:37:57.288618 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:57.288622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:57.288703 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:57.288788 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:57.289650 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:57.289654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:57.289657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:57.289670 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:57.289700 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:57.289711 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:57.289815 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:57.289895 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 19:37:57.289941 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:37:57.290900 19703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:57.291002 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:57.290596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:45485 (local address 127.0.0.1:46780)
0807 19:37:57.290767 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:57.290867 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:45485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 19:37:57.292106 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:57.291689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:45485 (local address 127.0.0.1:46794)
0807 19:37:57.291854 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:57.291960 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:45485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 19:37:57.372542 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:57.372052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:45485 (local address 127.0.0.1:46802)
0807 19:37:57.372278 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:57.372379 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:45485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 19:37:57.373648 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:57.373252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:45485 (local address 127.0.0.1:46818)
0807 19:37:57.373463 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:57.373536 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:45485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 19:37:57.380445 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 19580
I20260807 19:37:57.401176 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 19221
I20260807 19:37:57.402302 19283 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:57.509553 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 19221
2026-08-07T19:37:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2587 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 19:37:57.554462 14401 mini_oidc.cc:152] Starting JWKS server
I20260807 19:37:57.557196 14401 webserver.cc:460] Webserver started at https://127.0.0.1:38307/ using document root <none> and password file <none>
I20260807 19:37:57.557565 14401 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 19:37:57.558238 14401 webserver.cc:460] Webserver started at http://127.0.0.1:34791/ using document root <none> and password file <none>
2026-08-07T19:37:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:57Z Disabled control of system clock
I20260807 19:37:57.596933 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:42409
--webserver_interface=127.14.16.126
--webserver_port=0
--builtin_ntp_servers=127.14.16.84:34959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:42409
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38307/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/testchainca.cert with env {}
W20260807 19:37:57.921828 19766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:57.922109 19766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:57.922199 19766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:57.929316 19766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:57.929414 19766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:57.929463 19766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:57.929520 19766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:57.929565 19766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:37:57.929620 19766 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:37:57.929661 19766 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38307/jwks.json
I20260807 19:37:57.939494 19766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:34959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:42409
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:42409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38307/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:57.940716 19766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:57.941972 19766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:57.955574 19775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:57.955675 19772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:57.955654 19773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:57.956459 19766 server_base.cc:1034] running on GCE node
I20260807 19:37:57.957279 19766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:57.958734 19766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:57.959960 19766 hybrid_clock.cc:648] HybridClock initialized: now 1786131477959919 us; error 52 us; skew 500 ppm
I20260807 19:37:57.960325 19766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:37:57.962488 19766 webserver.cc:460] Webserver started at http://127.14.16.126:35403/ using document root <none> and password file <none>
I20260807 19:37:57.963156 19766 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:57.963328 19766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:57.963649 19766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:57.966467 19766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "9864c8cfed504bdba8b91849fe18ec07"
format_stamp: "Formatted at 2026-08-07 19:37:57 on dist-test-slave-07c2"
server_key: "203b42f3e01011a3be29cd84d6d9049e"
server_key_iv: "7b1843dd60e4d134c594036ddf4a1a4b"
server_key_version: "encryptionkey@0"
I20260807 19:37:57.967247 19766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "9864c8cfed504bdba8b91849fe18ec07"
format_stamp: "Formatted at 2026-08-07 19:37:57 on dist-test-slave-07c2"
server_key: "203b42f3e01011a3be29cd84d6d9049e"
server_key_iv: "7b1843dd60e4d134c594036ddf4a1a4b"
server_key_version: "encryptionkey@0"
I20260807 19:37:57.972235 19766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 19:37:57.975435 19781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:57.976584 19766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 19:37:57.976774 19766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "9864c8cfed504bdba8b91849fe18ec07"
format_stamp: "Formatted at 2026-08-07 19:37:57 on dist-test-slave-07c2"
server_key: "203b42f3e01011a3be29cd84d6d9049e"
server_key_iv: "7b1843dd60e4d134c594036ddf4a1a4b"
server_key_version: "encryptionkey@0"
I20260807 19:37:57.976982 19766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:58.002767 19766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:58.003646 19766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:58.003964 19766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:58.024398 19766 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:42409
I20260807 19:37:58.024410 19833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:42409 every 8 connection(s)
I20260807 19:37:58.025947 19766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:58.030445 19834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:58.031044 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 19766
I20260807 19:37:58.031275 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:58.031935 14401 external_mini_cluster.cc:1428] Setting key 0a1168d9ca3a3b899403e7aefcf32eb4
I20260807 19:37:58.044083 19834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07: Bootstrap starting.
I20260807 19:37:58.045722 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:58.033860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45690 (local address 127.14.16.126:42409)
0807 19:37:58.034915 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:58.034925 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:58.034969 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:37:58.036592 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:58.036615 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:58.037060 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:58.037391 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:58.038020 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:58.039472 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:58.041454 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:58.042110 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:58.042257 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:58.043508 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:58.043515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:58.043528 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:58.044057 (+ 529us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:58.044135 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:58.044159 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:58.044305 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:58.044519 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 19:37:58.044822 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":402,"threads_started":1}
I20260807 19:37:58.047576 19834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:58.048964 19834 log.cc:824] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:58.049939 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:37:58.051728 19834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07: No bootstrap required, opened a new log
I20260807 19:37:58.058936 19834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9864c8cfed504bdba8b91849fe18ec07" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42409 } }
I20260807 19:37:58.059396 19834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:58.059500 19834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9864c8cfed504bdba8b91849fe18ec07, State: Initialized, Role: FOLLOWER
I20260807 19:37:58.060026 19834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9864c8cfed504bdba8b91849fe18ec07" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42409 } }
I20260807 19:37:58.060268 19834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:58.060418 19834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:58.060593 19834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:58.062668 19834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9864c8cfed504bdba8b91849fe18ec07" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42409 } }
I20260807 19:37:58.063136 19834 leader_election.cc:302] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9864c8cfed504bdba8b91849fe18ec07; no voters:
I20260807 19:37:58.063735 19834 leader_election.cc:288] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:58.063891 19839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:58.065073 19839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [term 1 LEADER]: Becoming Leader. State: Replica: 9864c8cfed504bdba8b91849fe18ec07, State: Running, Role: LEADER
I20260807 19:37:58.065598 19839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9864c8cfed504bdba8b91849fe18ec07" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42409 } }
I20260807 19:37:58.065917 19834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:58.068876 19841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9864c8cfed504bdba8b91849fe18ec07. Latest consensus state: current_term: 1 leader_uuid: "9864c8cfed504bdba8b91849fe18ec07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9864c8cfed504bdba8b91849fe18ec07" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42409 } } }
I20260807 19:37:58.068886 19840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9864c8cfed504bdba8b91849fe18ec07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9864c8cfed504bdba8b91849fe18ec07" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42409 } } }
I20260807 19:37:58.069566 19841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:58.069702 19840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:58.070852 19847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:58.076375 19847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:58.085054 19847 catalog_manager.cc:1347] Generated new cluster ID: 1573ea68592f497aa73da768c27257ca
I20260807 19:37:58.085186 19847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:58.109140 19847 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:58.110424 19847 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:58.119403 19847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9864c8cfed504bdba8b91849fe18ec07: Generated new TSK 0
I20260807 19:37:58.120489 19847 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 19:37:58.164124 19866 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 19:37:58.164332 19866 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 19:37:58.165373 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:58.143793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.14.16.126:42409)
0807 19:37:58.144046 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:58.144053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:58.144507 (+ 454us) server_negotiation.cc:407] Connection header received
0807 19:37:58.144845 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:58.144850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:58.144944 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:58.145081 (+ 137us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:37:58.145745 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:58.146566 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:58.148789 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:58.149263 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:58.149366 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:58.149982 (+ 616us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:37:58.164842 (+ 14860us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 19:37:58.165135 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45704: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 19:37:58.165516 19837 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45704: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 19:37:58.167526 19865 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:58.143690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:42409 (local address 127.0.0.1:45704)
0807 19:37:58.144374 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:58.144395 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:58.144587 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:58.145141 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:58.145147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:58.145163 (+ 16us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:37:58.145578 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:58.145591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:58.146718 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:58.146731 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:58.148613 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:58.148629 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:58.149455 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:58.149462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:58.149651 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:58.149735 (+ 84us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:37:58.165165 (+ 15430us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 19:37:58.165545 (+ 380us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.16.126:42409: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":315,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.14.16.126:42409: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 19:37:58.169731 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 19766
I20260807 19:37:58.170872 19829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:58.269348 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 19766
2026-08-07T19:37:58Z chronyd exiting
I20260807 19:37:58.318832 14401 test_util.cc:175] -----------------------------------------------
I20260807 19:37:58.318994 14401 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131435615921-14401-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (770 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 19:37:58.322654 14401 mini_oidc.cc:152] Starting JWKS server
I20260807 19:37:58.324538 14401 webserver.cc:460] Webserver started at https://127.0.0.1:43057/ using document root <none> and password file <none>
I20260807 19:37:58.324781 14401 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 19:37:58.325425 14401 webserver.cc:460] Webserver started at http://127.0.0.1:46079/ using document root <none> and password file <none>
2026-08-07T19:37:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:58Z Disabled control of system clock
I20260807 19:37:58.356925 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:46445
--webserver_interface=127.14.16.126
--webserver_port=0
--builtin_ntp_servers=127.14.16.84:37863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:46445
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43057/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/testchainca.cert with env {}
W20260807 19:37:58.688625 19873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:58.689025 19873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:58.689210 19873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:58.697526 19873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:58.697715 19873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:58.697817 19873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:58.697952 19873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:58.698040 19873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:37:58.698153 19873 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:37:58.698242 19873 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43057/jwks.json
I20260807 19:37:58.709547 19873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:37863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:46445
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:46445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43057/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:58.710958 19873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:58.712404 19873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:58.724332 19878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:58.724356 19879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:58.724418 19881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:58.725625 19873 server_base.cc:1034] running on GCE node
I20260807 19:37:58.726480 19873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:58.728005 19873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:58.729242 19873 hybrid_clock.cc:648] HybridClock initialized: now 1786131478729205 us; error 33 us; skew 500 ppm
I20260807 19:37:58.729604 19873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:37:58.736503 19873 webserver.cc:460] Webserver started at http://127.14.16.126:36253/ using document root <none> and password file <none>
I20260807 19:37:58.737286 19873 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:58.737432 19873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:58.737783 19873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:58.740867 19873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "3a667a4d388649b3a9fdde5707e7fc43"
format_stamp: "Formatted at 2026-08-07 19:37:58 on dist-test-slave-07c2"
server_key: "9ae06ae06fbea85073511d100b1a00e2"
server_key_iv: "98570f72d05f9a1dc8f4931db377993d"
server_key_version: "encryptionkey@0"
I20260807 19:37:58.741693 19873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "3a667a4d388649b3a9fdde5707e7fc43"
format_stamp: "Formatted at 2026-08-07 19:37:58 on dist-test-slave-07c2"
server_key: "9ae06ae06fbea85073511d100b1a00e2"
server_key_iv: "98570f72d05f9a1dc8f4931db377993d"
server_key_version: "encryptionkey@0"
I20260807 19:37:58.747028 19873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:37:58.750649 19887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:58.752089 19873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:37:58.752287 19873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "3a667a4d388649b3a9fdde5707e7fc43"
format_stamp: "Formatted at 2026-08-07 19:37:58 on dist-test-slave-07c2"
server_key: "9ae06ae06fbea85073511d100b1a00e2"
server_key_iv: "98570f72d05f9a1dc8f4931db377993d"
server_key_version: "encryptionkey@0"
I20260807 19:37:58.752514 19873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:58.781888 19873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:58.782763 19873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:58.783082 19873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:58.801906 19873 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:46445
I20260807 19:37:58.801926 19939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:46445 every 8 connection(s)
I20260807 19:37:58.803231 19873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:58.806329 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 19873
I20260807 19:37:58.806538 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:58.806896 19941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:58.807312 14401 external_mini_cluster.cc:1428] Setting key b0ca40ca4594827a597b373a21302ac8
I20260807 19:37:58.819552 19941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43: Bootstrap starting.
I20260807 19:37:58.820297 19944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:58.809148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46006 (local address 127.14.16.126:46445)
0807 19:37:58.810112 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:58.810123 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:58.810166 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:37:58.811831 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:58.811852 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:58.812291 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:58.812686 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:58.813266 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:58.814528 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:58.815850 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:58.816530 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:58.816743 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:58.817934 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:58.817941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:58.817953 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:58.818462 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:58.818532 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:58.818567 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:58.818702 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:58.818916 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 19:37:58.819198 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":345,"threads_started":1}
I20260807 19:37:58.822715 19941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:58.823921 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:37:58.823968 19941 log.cc:824] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:58.826818 19941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43: No bootstrap required, opened a new log
I20260807 19:37:58.834336 19941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a667a4d388649b3a9fdde5707e7fc43" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46445 } }
I20260807 19:37:58.834674 19941 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:58.834824 19941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a667a4d388649b3a9fdde5707e7fc43, State: Initialized, Role: FOLLOWER
I20260807 19:37:58.835506 19941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a667a4d388649b3a9fdde5707e7fc43" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46445 } }
I20260807 19:37:58.835727 19941 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:58.835872 19941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:58.836037 19941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:58.838490 19941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a667a4d388649b3a9fdde5707e7fc43" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46445 } }
I20260807 19:37:58.839066 19941 leader_election.cc:302] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a667a4d388649b3a9fdde5707e7fc43; no voters:
I20260807 19:37:58.839653 19941 leader_election.cc:288] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:58.839802 19946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:58.840606 19946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [term 1 LEADER]: Becoming Leader. State: Replica: 3a667a4d388649b3a9fdde5707e7fc43, State: Running, Role: LEADER
I20260807 19:37:58.841202 19946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a667a4d388649b3a9fdde5707e7fc43" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46445 } }
I20260807 19:37:58.841907 19941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:58.844622 19948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a667a4d388649b3a9fdde5707e7fc43. Latest consensus state: current_term: 1 leader_uuid: "3a667a4d388649b3a9fdde5707e7fc43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a667a4d388649b3a9fdde5707e7fc43" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46445 } } }
I20260807 19:37:58.844589 19947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a667a4d388649b3a9fdde5707e7fc43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a667a4d388649b3a9fdde5707e7fc43" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46445 } } }
I20260807 19:37:58.845104 19947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:58.845259 19948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43 [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:58.847491 19951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:58.852655 19951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:58.860872 19951 catalog_manager.cc:1347] Generated new cluster ID: d030322879f2497bbabdbe7d8c99d15d
I20260807 19:37:58.860978 19951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:58.887241 19951 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:58.888700 19951 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:58.899412 19951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a667a4d388649b3a9fdde5707e7fc43: Generated new TSK 0
I20260807 19:37:58.900200 19951 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 19:37:58.941110 19971 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 19:37:58.941252 19971 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 19:37:58.942430 19944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:58.915456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.14.16.126:46445)
0807 19:37:58.915718 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:58.915733 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:58.916104 (+ 371us) server_negotiation.cc:407] Connection header received
0807 19:37:58.916335 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:58.916339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:58.916409 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:58.916546 (+ 137us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:37:58.917107 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:58.917839 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:58.919235 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:58.919744 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:58.919890 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:58.920373 (+ 483us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:37:58.941627 (+ 21254us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43057/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 19:37:58.942074 (+ 447us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46014: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43057/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 19:37:58.942569 19944 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46014: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43057/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 19:37:58.942771 19970 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:58.915341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:46445 (local address 127.0.0.1:46014)
0807 19:37:58.915986 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:58.916008 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:58.916164 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:58.916586 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:58.916592 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:58.916608 (+ 16us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:37:58.916932 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:58.916946 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:58.918000 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:58.918007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:58.919085 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:58.919096 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:58.919909 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:58.919915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:58.920075 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:58.920109 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:37:58.941876 (+ 21767us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43057/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 19:37:58.942184 (+ 308us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.16.126:46445: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43057/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":342,"threads_started":1}
I20260807 19:37:58.944283 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 19873
I20260807 19:37:58.945567 19935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:59.040089 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 19873
2026-08-07T19:37:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (775 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 19:37:59.097913 14401 mini_oidc.cc:152] Starting JWKS server
I20260807 19:37:59.099826 14401 webserver.cc:460] Webserver started at https://127.0.0.1:36877/ using document root <none> and password file <none>
I20260807 19:37:59.100029 14401 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 19:37:59.100641 14401 webserver.cc:460] Webserver started at http://127.0.0.1:42985/ using document root <none> and password file <none>
2026-08-07T19:37:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:59Z Disabled control of system clock
I20260807 19:37:59.133798 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:44771
--webserver_interface=127.14.16.126
--webserver_port=0
--builtin_ntp_servers=127.14.16.84:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:44771
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36877/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 19:37:59.456202 19978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:59.456466 19978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:59.456684 19978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:59.463624 19978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:59.463711 19978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:59.463806 19978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:59.463888 19978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:59.463964 19978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:37:59.464064 19978 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:37:59.464120 19978 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36877/jwks.json
I20260807 19:37:59.473014 19978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:44771
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:44771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36877/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:59.474202 19978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:59.475960 19978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:59.487401 19984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:59.487493 19985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:59.487493 19987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:59.488739 19978 server_base.cc:1034] running on GCE node
I20260807 19:37:59.489528 19978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:59.490877 19978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:59.492097 19978 hybrid_clock.cc:648] HybridClock initialized: now 1786131479492063 us; error 55 us; skew 500 ppm
I20260807 19:37:59.492441 19978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:37:59.494735 19978 webserver.cc:460] Webserver started at http://127.14.16.126:43729/ using document root <none> and password file <none>
I20260807 19:37:59.495467 19978 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:59.495592 19978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:59.495920 19978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:59.498754 19978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "f6e61f513ffa467aa5e2ac54e6a1026d"
format_stamp: "Formatted at 2026-08-07 19:37:59 on dist-test-slave-07c2"
server_key: "e67523aa1f6d4afa293c27e14df7864c"
server_key_iv: "7fe86b95c5455ec7c594c06695d6156e"
server_key_version: "encryptionkey@0"
I20260807 19:37:59.499590 19978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "f6e61f513ffa467aa5e2ac54e6a1026d"
format_stamp: "Formatted at 2026-08-07 19:37:59 on dist-test-slave-07c2"
server_key: "e67523aa1f6d4afa293c27e14df7864c"
server_key_iv: "7fe86b95c5455ec7c594c06695d6156e"
server_key_version: "encryptionkey@0"
I20260807 19:37:59.504558 19978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 19:37:59.507867 19993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:59.509176 19978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:37:59.509348 19978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "f6e61f513ffa467aa5e2ac54e6a1026d"
format_stamp: "Formatted at 2026-08-07 19:37:59 on dist-test-slave-07c2"
server_key: "e67523aa1f6d4afa293c27e14df7864c"
server_key_iv: "7fe86b95c5455ec7c594c06695d6156e"
server_key_version: "encryptionkey@0"
I20260807 19:37:59.509557 19978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:59.526943 19978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:59.527776 19978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:59.528066 19978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:59.546147 19978 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:44771
I20260807 19:37:59.546167 20045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:44771 every 8 connection(s)
I20260807 19:37:59.547495 19978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:59.551043 20046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:59.551899 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 19978
I20260807 19:37:59.552145 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:37:59.553025 14401 external_mini_cluster.cc:1428] Setting key cc5f0980354760d003160dcb67ddac66
I20260807 19:37:59.565387 20046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d: Bootstrap starting.
I20260807 19:37:59.567117 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:59.554972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.14.16.126:44771)
0807 19:37:59.555996 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:59.556010 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:59.556053 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:37:59.557688 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:59.557709 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:59.558150 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:59.558463 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:59.559058 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:59.560572 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:59.562512 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:59.563235 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:59.563456 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:59.564842 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:59.564850 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:59.564863 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:37:59.565409 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:59.565483 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:37:59.565504 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:59.565626 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:59.565887 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 19:37:59.566208 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":412,"threads_started":1}
I20260807 19:37:59.569072 20046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:59.570319 20046 log.cc:824] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:59.570914 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:37:59.574863 20046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d: No bootstrap required, opened a new log
I20260807 19:37:59.584039 20046 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e61f513ffa467aa5e2ac54e6a1026d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44771 } }
I20260807 19:37:59.584463 20046 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:37:59.584583 20046 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e61f513ffa467aa5e2ac54e6a1026d, State: Initialized, Role: FOLLOWER
I20260807 19:37:59.585204 20046 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e61f513ffa467aa5e2ac54e6a1026d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44771 } }
I20260807 19:37:59.585434 20046 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:37:59.585577 20046 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:37:59.585748 20046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:59.587961 20046 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e61f513ffa467aa5e2ac54e6a1026d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44771 } }
I20260807 19:37:59.588450 20046 leader_election.cc:302] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6e61f513ffa467aa5e2ac54e6a1026d; no voters:
I20260807 19:37:59.589057 20046 leader_election.cc:288] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:37:59.589238 20051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:59.590533 20051 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [term 1 LEADER]: Becoming Leader. State: Replica: f6e61f513ffa467aa5e2ac54e6a1026d, State: Running, Role: LEADER
I20260807 19:37:59.591193 20046 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:59.591034 20051 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e61f513ffa467aa5e2ac54e6a1026d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44771 } }
I20260807 19:37:59.597081 20053 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6e61f513ffa467aa5e2ac54e6a1026d. Latest consensus state: current_term: 1 leader_uuid: "f6e61f513ffa467aa5e2ac54e6a1026d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e61f513ffa467aa5e2ac54e6a1026d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44771 } } }
I20260807 19:37:59.597123 20052 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6e61f513ffa467aa5e2ac54e6a1026d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e61f513ffa467aa5e2ac54e6a1026d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44771 } } }
I20260807 19:37:59.597596 20053 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:59.597601 20052 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d [sys.catalog]: This master's current role is: LEADER
W20260807 19:37:59.601819 20066 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:37:59.601923 20066 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:37:59.602100 20068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:59.603716 20068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:59.611320 20068 catalog_manager.cc:1347] Generated new cluster ID: 73f1b0e84b9e4adaab5008cd5c7ff5fa
I20260807 19:37:59.611429 20068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:59.623086 20068 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:59.624368 20068 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:59.631991 20068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6e61f513ffa467aa5e2ac54e6a1026d: Generated new TSK 0
I20260807 19:37:59.632791 20068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:37:59.655869 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:59.647463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.14.16.126:44771)
0807 19:37:59.647893 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:59.647899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:59.648332 (+ 433us) server_negotiation.cc:407] Connection header received
0807 19:37:59.648525 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:59.648531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:59.648636 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:59.648818 (+ 182us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:37:59.649197 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:59.650297 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:59.651674 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:59.652347 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:59.652474 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:59.652953 (+ 479us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:37:59.655186 (+ 2233us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 19:37:59.655550 (+ 364us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36844: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":272}
W20260807 19:37:59.656980 20049 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36844: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 19:37:59.658224 20078 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:59.647564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44771 (local address 127.0.0.1:36844)
0807 19:37:59.648229 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:59.648248 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:59.648365 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:59.648810 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:59.648814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:59.648823 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:37:59.649058 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:59.649066 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:59.650423 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:59.650426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:59.651548 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:59.651559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:59.652518 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:59.652522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:59.652649 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:59.652679 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:37:59.655410 (+ 2731us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 19:37:59.657730 (+ 2320us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.16.126:44771: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36877/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":1961,"thread_start_us":364,"threads_started":1}
I20260807 19:37:59.659467 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 19978
I20260807 19:37:59.660678 20040 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 19:37:59.674139 20079 webserver.cc:533] Webserver: sslize failed:
E20260807 19:37:59.674273 20079 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 19:37:59.767787 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 19978
2026-08-07T19:37:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (727 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T19:37:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:59Z Disabled control of system clock
I20260807 19:37:59.860536 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:40385
--webserver_interface=127.14.16.126
--webserver_port=0
--builtin_ntp_servers=127.14.16.84:35119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:40385
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/testchainca.cert with env {}
W20260807 19:38:00.183684 20084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:00.183985 20084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:00.184124 20084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:00.191207 20084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:38:00.191347 20084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:00.191435 20084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:00.191534 20084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:38:00.191613 20084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:38:00.191702 20084 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:38:00.191782 20084 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 19:38:00.201490 20084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:35119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:40385
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:40385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:00.202739 20084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:00.204068 20084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:00.216679 20090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:00.216650 20091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:00.216811 20093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:00.217764 20084 server_base.cc:1034] running on GCE node
I20260807 19:38:00.218523 20084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:00.219964 20084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:00.221158 20084 hybrid_clock.cc:648] HybridClock initialized: now 1786131480221104 us; error 47 us; skew 500 ppm
I20260807 19:38:00.221498 20084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:38:00.223683 20084 webserver.cc:460] Webserver started at http://127.14.16.126:38515/ using document root <none> and password file <none>
I20260807 19:38:00.224356 20084 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:00.224493 20084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:00.224809 20084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:00.227650 20084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "8167650f143e45049b4c22621f1e10ba"
format_stamp: "Formatted at 2026-08-07 19:38:00 on dist-test-slave-07c2"
server_key: "be3c94fc4353aad9fbb297cdc213ac5c"
server_key_iv: "5560a0a98223037c5305887661f5a904"
server_key_version: "encryptionkey@0"
I20260807 19:38:00.228447 20084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "8167650f143e45049b4c22621f1e10ba"
format_stamp: "Formatted at 2026-08-07 19:38:00 on dist-test-slave-07c2"
server_key: "be3c94fc4353aad9fbb297cdc213ac5c"
server_key_iv: "5560a0a98223037c5305887661f5a904"
server_key_version: "encryptionkey@0"
I20260807 19:38:00.233405 20084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:38:00.236846 20099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:00.238189 20084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:38:00.238369 20084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "8167650f143e45049b4c22621f1e10ba"
format_stamp: "Formatted at 2026-08-07 19:38:00 on dist-test-slave-07c2"
server_key: "be3c94fc4353aad9fbb297cdc213ac5c"
server_key_iv: "5560a0a98223037c5305887661f5a904"
server_key_version: "encryptionkey@0"
I20260807 19:38:00.238579 20084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:00.254603 20084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:00.255524 20084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:00.255820 20084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:00.274998 20084 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:40385
I20260807 19:38:00.275022 20152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:40385 every 8 connection(s)
I20260807 19:38:00.276486 20084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:38:00.280368 20153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:00.281644 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 20084
I20260807 19:38:00.281879 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:38:00.282802 14401 external_mini_cluster.cc:1428] Setting key 9416bed6697980f3d198bde7e8398676
I20260807 19:38:00.295040 20153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba: Bootstrap starting.
I20260807 19:38:00.296243 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:00.284734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40818 (local address 127.14.16.126:40385)
0807 19:38:00.285758 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:00.285769 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:00.285812 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:38:00.287410 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:00.287431 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:00.287871 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:00.288268 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:00.288870 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:00.290086 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:00.291953 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:00.292618 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:00.292769 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:00.293957 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:00.293964 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:00.293976 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:00.294519 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:00.294594 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:00.294614 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:00.294698 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:00.294936 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 19:38:00.295373 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":378,"threads_started":1}
I20260807 19:38:00.298173 20153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:00.299368 20153 log.cc:824] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:00.299804 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:38:00.302407 20153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba: No bootstrap required, opened a new log
I20260807 19:38:00.309736 20153 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8167650f143e45049b4c22621f1e10ba" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40385 } }
I20260807 19:38:00.310058 20153 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:00.310199 20153 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8167650f143e45049b4c22621f1e10ba, State: Initialized, Role: FOLLOWER
I20260807 19:38:00.310712 20153 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8167650f143e45049b4c22621f1e10ba" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40385 } }
I20260807 19:38:00.310946 20153 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:38:00.311087 20153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:38:00.311254 20153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:00.313279 20153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8167650f143e45049b4c22621f1e10ba" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40385 } }
I20260807 19:38:00.313778 20153 leader_election.cc:302] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8167650f143e45049b4c22621f1e10ba; no voters:
I20260807 19:38:00.314401 20153 leader_election.cc:288] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:38:00.314519 20158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:00.315318 20158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [term 1 LEADER]: Becoming Leader. State: Replica: 8167650f143e45049b4c22621f1e10ba, State: Running, Role: LEADER
I20260807 19:38:00.315831 20158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8167650f143e45049b4c22621f1e10ba" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40385 } }
I20260807 19:38:00.316386 20153 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:38:00.318568 20160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8167650f143e45049b4c22621f1e10ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8167650f143e45049b4c22621f1e10ba" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40385 } } }
I20260807 19:38:00.318579 20161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8167650f143e45049b4c22621f1e10ba. Latest consensus state: current_term: 1 leader_uuid: "8167650f143e45049b4c22621f1e10ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8167650f143e45049b4c22621f1e10ba" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40385 } } }
I20260807 19:38:00.318957 20161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:00.318957 20160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:00.323065 20166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:38:00.328223 20166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:38:00.336591 20166 catalog_manager.cc:1347] Generated new cluster ID: da3ce08ce009470c882c2808cd1156fb
I20260807 19:38:00.336714 20166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:38:00.409838 20166 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:38:00.411695 20166 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:38:00.446609 20166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8167650f143e45049b4c22621f1e10ba: Generated new TSK 0
I20260807 19:38:00.447536 20166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:38:00.480645 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:00.461697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.14.16.126:40385)
0807 19:38:00.461941 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:00.461950 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:00.462417 (+ 467us) server_negotiation.cc:407] Connection header received
0807 19:38:00.462638 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:00.462642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:00.462710 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:00.462851 (+ 141us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:38:00.463391 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:00.464128 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:00.466171 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:00.466606 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:00.466695 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:00.467242 (+ 547us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:38:00.480090 (+ 12848us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 19:38:00.480402 (+ 312us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40830: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 19:38:00.480832 20156 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40830: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 19:38:00.481093 20183 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:00.461670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40385 (local address 127.0.0.1:40830)
0807 19:38:00.462310 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:00.462330 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:00.462484 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:00.462891 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:00.462898 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:00.462915 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:38:00.463233 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:00.463246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:00.464278 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:00.464284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:00.466032 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:00.466054 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:00.466772 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:00.466778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:00.466958 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:00.467002 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:38:00.480354 (+ 13352us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 19:38:00.480663 (+ 309us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.16.126:40385: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":346,"threads_started":1}
I20260807 19:38:00.483031 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 20084
I20260807 19:38:00.484174 20148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:00.583065 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 20084
2026-08-07T19:38:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (794 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (409 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (412 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:38:01 dist-test-slave-07c2 krb5kdc[20198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:38:01 dist-test-slave-07c2 krb5kdc[20198](info): set up 2 sockets
Aug 07 19:38:01 dist-test-slave-07c2 krb5kdc[20198](info): commencing operation
krb5kdc: starting...
W20260807 19:38:02.526641 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:38:02 dist-test-slave-07c2 krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:38:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:38:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:38:02.771386 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:43653
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:43653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:03.094817 20211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:03.095085 20211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:03.095177 20211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:03.102130 20211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:38:03.102283 20211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:03.102348 20211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:03.102408 20211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:38:03.102450 20211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:38:03.111951 20211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:43653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:43653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:03.113225 20211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:03.114437 20211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:03.126085 20219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:03.126168 20216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:03.126140 20217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:03.126876 20211 server_base.cc:1034] running on GCE node
I20260807 19:38:03.127676 20211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:03.128944 20211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:03.130141 20211 hybrid_clock.cc:648] HybridClock initialized: now 1786131483130107 us; error 29 us; skew 500 ppm
I20260807 19:38:03.130471 20211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:03 dist-test-slave-07c2 krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131483, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:03.133407 20211 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:38:03.134829 20211 webserver.cc:460] Webserver started at http://127.14.16.126:46279/ using document root <none> and password file <none>
I20260807 19:38:03.135535 20211 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:03.135677 20211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:03.135984 20211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:03.138800 20211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "fc6a3ad5c8924411ae225e1f590db531"
format_stamp: "Formatted at 2026-08-07 19:38:03 on dist-test-slave-07c2"
server_key: "9078bb152650137e34dca3484e4de966"
server_key_iv: "15400de87c3d64eb26019ed9ee7b99ec"
server_key_version: "encryptionkey@0"
I20260807 19:38:03.139608 20211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "fc6a3ad5c8924411ae225e1f590db531"
format_stamp: "Formatted at 2026-08-07 19:38:03 on dist-test-slave-07c2"
server_key: "9078bb152650137e34dca3484e4de966"
server_key_iv: "15400de87c3d64eb26019ed9ee7b99ec"
server_key_version: "encryptionkey@0"
I20260807 19:38:03.144515 20211 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:38:03.147747 20227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:03.148912 20211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:38:03.149092 20211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "fc6a3ad5c8924411ae225e1f590db531"
format_stamp: "Formatted at 2026-08-07 19:38:03 on dist-test-slave-07c2"
server_key: "9078bb152650137e34dca3484e4de966"
server_key_iv: "15400de87c3d64eb26019ed9ee7b99ec"
server_key_version: "encryptionkey@0"
I20260807 19:38:03.149281 20211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:03.165325 20211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:03.169251 20211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:03.169535 20211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:03.188104 20211 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:43653
I20260807 19:38:03.188139 20279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:43653 every 8 connection(s)
I20260807 19:38:03.189500 20211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:38:03.193045 20280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:03.194119 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 20211
I20260807 19:38:03.194375 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:38:03.195318 14401 external_mini_cluster.cc:1428] Setting key ba52913f0c7a39541ef689626467c34c
I20260807 19:38:03.207466 20280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531: Bootstrap starting.
Aug 07 19:38:03 dist-test-slave-07c2 krb5kdc[20198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:38:03.210928 20280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:03.212046 20280 log.cc:824] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:03.215252 20280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531: No bootstrap required, opened a new log
I20260807 19:38:03.218082 20284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:03.197323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.14.16.126:43653)
0807 19:38:03.198141 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:03.198152 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:03.198190 (+ 38us) server_negotiation.cc:407] Connection header received
0807 19:38:03.199836 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:03.199856 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:03.200249 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:03.200562 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:03.202225 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:03.203509 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:03.205276 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:03.205893 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:03.206016 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:03.210564 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:03.210570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:03.210584 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:03.210618 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:03.214124 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:03.214797 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:03.214805 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:03.214813 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:03.215025 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:03.215390 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:03.215396 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:03.215399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:03.215903 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:03.216255 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:03.216651 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 19:38:03.217102 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":379,"threads_started":1}
I20260807 19:38:03.223735 20280 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6a3ad5c8924411ae225e1f590db531" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43653 } }
I20260807 19:38:03.224208 20280 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:03.224349 20280 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6a3ad5c8924411ae225e1f590db531, State: Initialized, Role: FOLLOWER
I20260807 19:38:03.225000 20280 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6a3ad5c8924411ae225e1f590db531" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43653 } }
I20260807 19:38:03.225306 20280 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:38:03.225445 20280 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:38:03.225606 20280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:03.227990 20280 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6a3ad5c8924411ae225e1f590db531" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43653 } }
I20260807 19:38:03.228592 20280 leader_election.cc:302] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc6a3ad5c8924411ae225e1f590db531; no voters:
I20260807 19:38:03.229152 20280 leader_election.cc:288] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:38:03.229281 20286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:03.229987 20286 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [term 1 LEADER]: Becoming Leader. State: Replica: fc6a3ad5c8924411ae225e1f590db531, State: Running, Role: LEADER
I20260807 19:38:03.230499 20286 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6a3ad5c8924411ae225e1f590db531" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43653 } }
I20260807 19:38:03.231231 20280 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:38:03.233223 20288 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc6a3ad5c8924411ae225e1f590db531. Latest consensus state: current_term: 1 leader_uuid: "fc6a3ad5c8924411ae225e1f590db531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6a3ad5c8924411ae225e1f590db531" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43653 } } }
I20260807 19:38:03.233251 20287 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc6a3ad5c8924411ae225e1f590db531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6a3ad5c8924411ae225e1f590db531" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43653 } } }
I20260807 19:38:03.233727 20287 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:03.233727 20288 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:03.237629 20295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:38:03.242084 20295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:38:03.250316 20295 catalog_manager.cc:1347] Generated new cluster ID: 41afa199dce149769806f2851f18fcfc
I20260807 19:38:03.250422 20295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:38:03.277931 20295 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:38:03.279862 20295 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:38:03.299710 20295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc6a3ad5c8924411ae225e1f590db531: Generated new TSK 0
I20260807 19:38:03.300719 20295 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:38:03.356088 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43653
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:03.693362 20308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:03.693619 20308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:03.693717 20308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:03.700829 20308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:03.700927 20308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:03.701035 20308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:38:03.710415 20308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43653
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:03.711592 20308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:03.712782 20308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:03.725726 20313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:03.725791 20316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:03.725826 20314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:03.726758 20308 server_base.cc:1034] running on GCE node
I20260807 19:38:03.727334 20308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:03.728166 20308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:03.729359 20308 hybrid_clock.cc:648] HybridClock initialized: now 1786131483729311 us; error 35 us; skew 500 ppm
I20260807 19:38:03.729653 20308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:03 dist-test-slave-07c2 krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131483, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:03.732923 20308 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:38:03.734244 20308 webserver.cc:460] Webserver started at http://127.14.16.65:38691/ using document root <none> and password file <none>
I20260807 19:38:03.734931 20308 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:03.735061 20308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:03.735430 20308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:03.738220 20308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "18396a1b05b04dd9b390c0ae51ed7543"
format_stamp: "Formatted at 2026-08-07 19:38:03 on dist-test-slave-07c2"
server_key: "c23ca54d854df27bded9bf31dbd32717"
server_key_iv: "503d65f1477b60de4e61b6bb1749a47f"
server_key_version: "encryptionkey@0"
I20260807 19:38:03.738993 20308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "18396a1b05b04dd9b390c0ae51ed7543"
format_stamp: "Formatted at 2026-08-07 19:38:03 on dist-test-slave-07c2"
server_key: "c23ca54d854df27bded9bf31dbd32717"
server_key_iv: "503d65f1477b60de4e61b6bb1749a47f"
server_key_version: "encryptionkey@0"
I20260807 19:38:03.744123 20308 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 19:38:03.748783 20323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:03.750427 20308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260807 19:38:03.750608 20308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "18396a1b05b04dd9b390c0ae51ed7543"
format_stamp: "Formatted at 2026-08-07 19:38:03 on dist-test-slave-07c2"
server_key: "c23ca54d854df27bded9bf31dbd32717"
server_key_iv: "503d65f1477b60de4e61b6bb1749a47f"
server_key_version: "encryptionkey@0"
I20260807 19:38:03.750886 20308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:03.780033 20308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:03.784346 20308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:03.784581 20308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:03.785468 20308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:03.786669 20308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:03.786744 20308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:03.786835 20308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:03.786885 20308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:03.818976 20308 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:34365
I20260807 19:38:03.819061 20439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:34365 every 8 connection(s)
I20260807 19:38:03.820276 20308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:38:03.828325 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 20308
I20260807 19:38:03.828577 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:38:03.829304 14401 external_mini_cluster.cc:1428] Setting key e8168f67af67d851f4f3951bf1f90d3d
Aug 07 19:38:03 dist-test-slave-07c2 krb5kdc[20198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131483, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:38:03.840858 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:03.822764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:37891 (local address 127.14.16.126:43653)
0807 19:38:03.823183 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:03.823188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:03.824201 (+ 1013us) server_negotiation.cc:407] Connection header received
0807 19:38:03.825991 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:03.825998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:03.826102 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:03.826217 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:03.827735 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:03.828488 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:03.830193 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:03.830772 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:03.830904 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:03.835092 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:03.835099 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:03.835108 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:03.835140 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:03.837760 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:03.838492 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:03.838497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:03.838505 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:03.838624 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:03.839060 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:03.839065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:03.839068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:03.839392 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:03.839541 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:03.840250 (+ 709us) server_negotiation.cc:299] Negotiation successful
0807 19:38:03.840480 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":217,"threads_started":1}
I20260807 19:38:03.841847 20443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:03.823352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:43653 (local address 127.14.16.65:37891)
0807 19:38:03.824049 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:03.824085 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:03.825728 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:03.826445 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:03.826452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:03.826944 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:03.827573 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:03.827592 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:03.828671 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:03.828679 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:03.830049 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:03.830066 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:03.831015 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:03.831025 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:03.831325 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:03.832138 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:03.832169 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:03.834872 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:03.837918 (+ 3046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:03.837927 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:03.837943 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:03.838367 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:03.838735 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:03.838740 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:03.838743 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:03.838940 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:03.839525 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:03.839533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:03.839862 (+ 329us) client_negotiation.cc:769] Sending connection context
0807 19:38:03.840132 (+ 270us) client_negotiation.cc:240] Negotiation successful
0807 19:38:03.840675 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":277,"threads_started":1}
I20260807 19:38:03.843528 20440 heartbeater.cc:347] Connected to a master server at 127.14.16.126:43653
I20260807 19:38:03.843847 20440 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:03.844533 20440 heartbeater.cc:511] Master 127.14.16.126:43653 requested a full tablet report, sending...
I20260807 19:38:03.846606 20245 ts_manager.cc:194] Registered new tserver with Master: 18396a1b05b04dd9b390c0ae51ed7543 (127.14.16.65:34365)
I20260807 19:38:03.848093 20245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:37891
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:38:03.883706 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43653
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:04.217917 20446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:04.218199 20446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:04.218348 20446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:04.225544 20446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:04.225641 20446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:04.225852 20446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:38:04.236734 20446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43653
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:04.237934 20446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:04.239212 20446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:04.251857 20454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:04.252004 20451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:04.252167 20452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:04.253106 20446 server_base.cc:1034] running on GCE node
I20260807 19:38:04.253757 20446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:04.254683 20446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:04.255911 20446 hybrid_clock.cc:648] HybridClock initialized: now 1786131484255862 us; error 53 us; skew 500 ppm
I20260807 19:38:04.256258 20446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:04.259451 20446 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:38:04.260941 20446 webserver.cc:460] Webserver started at http://127.14.16.66:44547/ using document root <none> and password file <none>
I20260807 19:38:04.261607 20446 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:04.261754 20446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:04.262058 20446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:04.265054 20446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "d720d7d8ba9145128273113f6698225e"
format_stamp: "Formatted at 2026-08-07 19:38:04 on dist-test-slave-07c2"
server_key: "bfcb34382e7b67c3e8cbcf09e77e4ad6"
server_key_iv: "ef64e69f9bac117316e1a14c977af153"
server_key_version: "encryptionkey@0"
I20260807 19:38:04.265880 20446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "d720d7d8ba9145128273113f6698225e"
format_stamp: "Formatted at 2026-08-07 19:38:04 on dist-test-slave-07c2"
server_key: "bfcb34382e7b67c3e8cbcf09e77e4ad6"
server_key_iv: "ef64e69f9bac117316e1a14c977af153"
server_key_version: "encryptionkey@0"
I20260807 19:38:04.270813 20446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 19:38:04.274219 20461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:04.275543 20446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:38:04.275717 20446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "d720d7d8ba9145128273113f6698225e"
format_stamp: "Formatted at 2026-08-07 19:38:04 on dist-test-slave-07c2"
server_key: "bfcb34382e7b67c3e8cbcf09e77e4ad6"
server_key_iv: "ef64e69f9bac117316e1a14c977af153"
server_key_version: "encryptionkey@0"
I20260807 19:38:04.275910 20446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:04.297611 20446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:04.301640 20446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:04.301936 20446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:04.302887 20446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:04.304248 20446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:04.304324 20446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:04.304479 20446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:04.304534 20446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:04.337291 20446 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:44697
I20260807 19:38:04.337345 20575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:44697 every 8 connection(s)
I20260807 19:38:04.338518 20446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:38:04.346014 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 20446
I20260807 19:38:04.346197 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:38:04.346653 14401 external_mini_cluster.cc:1428] Setting key 95e11e1204514de9c2e1e523cd5460fc
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:38:04.359797 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:04.341019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:36601 (local address 127.14.16.126:43653)
0807 19:38:04.341202 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:04.341206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:04.342313 (+ 1107us) server_negotiation.cc:407] Connection header received
0807 19:38:04.344239 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:04.344246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:04.344347 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:04.344453 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:04.345887 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.346697 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.348486 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.349126 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.349272 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:04.352710 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:04.352720 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:04.352727 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:04.352772 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:04.356526 (+ 3754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:04.357290 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:04.357298 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:04.357304 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:04.357463 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:04.357875 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:04.357882 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:04.357887 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:04.358160 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:04.358269 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:04.359393 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0807 19:38:04.359560 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:38:04.361402 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:04.341417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:43653 (local address 127.14.16.66:36601)
0807 19:38:04.342156 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:04.342196 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:04.343977 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:04.344674 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:04.344681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:04.345174 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:04.345710 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:04.345732 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.346882 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.346888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:04.348293 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:04.348306 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.349334 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.349340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:04.349528 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:04.350191 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:04.350215 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:04.352423 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:04.356653 (+ 4230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:04.356659 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:04.356671 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:04.357168 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:04.357567 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:04.357570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:04.357572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:04.357744 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:04.358284 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:04.358294 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:04.358726 (+ 432us) client_negotiation.cc:769] Sending connection context
0807 19:38:04.359249 (+ 523us) client_negotiation.cc:240] Negotiation successful
0807 19:38:04.359837 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":287,"threads_started":1}
I20260807 19:38:04.362942 20576 heartbeater.cc:347] Connected to a master server at 127.14.16.126:43653
I20260807 19:38:04.363365 20576 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:04.364063 20576 heartbeater.cc:511] Master 127.14.16.126:43653 requested a full tablet report, sending...
I20260807 19:38:04.365561 20245 ts_manager.cc:194] Registered new tserver with Master: d720d7d8ba9145128273113f6698225e (127.14.16.66:44697)
I20260807 19:38:04.366384 20245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:36601
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:38:04.400434 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43653
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:04.733978 20582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:04.734300 20582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:04.734465 20582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:04.741797 20582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:04.741919 20582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:04.742094 20582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:38:04.751638 20582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43653
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:04.752884 20582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:04.754345 20582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:04.767504 20588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:04.767599 20587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:04.767583 20590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:04.768123 20582 server_base.cc:1034] running on GCE node
I20260807 19:38:04.768692 20582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:04.769590 20582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:04.770820 20582 hybrid_clock.cc:648] HybridClock initialized: now 1786131484770782 us; error 56 us; skew 500 ppm
I20260807 19:38:04.771136 20582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:04.774128 20582 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:38:04.775557 20582 webserver.cc:460] Webserver started at http://127.14.16.67:37625/ using document root <none> and password file <none>
I20260807 19:38:04.776176 20582 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:04.776316 20582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:04.776610 20582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:04.779464 20582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "15040874938a452ea604d1f1dbbf07a6"
format_stamp: "Formatted at 2026-08-07 19:38:04 on dist-test-slave-07c2"
server_key: "433532926fb4e314bb74d8dd06f4b455"
server_key_iv: "25aad69b90888aa776677824b9e756bf"
server_key_version: "encryptionkey@0"
I20260807 19:38:04.780264 20582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "15040874938a452ea604d1f1dbbf07a6"
format_stamp: "Formatted at 2026-08-07 19:38:04 on dist-test-slave-07c2"
server_key: "433532926fb4e314bb74d8dd06f4b455"
server_key_iv: "25aad69b90888aa776677824b9e756bf"
server_key_version: "encryptionkey@0"
I20260807 19:38:04.785208 20582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:38:04.788381 20597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:04.789672 20582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:38:04.789832 20582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "15040874938a452ea604d1f1dbbf07a6"
format_stamp: "Formatted at 2026-08-07 19:38:04 on dist-test-slave-07c2"
server_key: "433532926fb4e314bb74d8dd06f4b455"
server_key_iv: "25aad69b90888aa776677824b9e756bf"
server_key_version: "encryptionkey@0"
I20260807 19:38:04.790011 20582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:04.808072 20582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:04.812139 20582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:04.812422 20582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:04.813318 20582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:04.814726 20582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:04.814805 20582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:04.814986 20582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:04.815073 20582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:04.849943 20582 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:44173
I20260807 19:38:04.849999 20714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:44173 every 8 connection(s)
I20260807 19:38:04.850836 20440 heartbeater.cc:503] Master 127.14.16.126:43653 was elected leader, sending a full tablet report...
I20260807 19:38:04.851426 20582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:38:04.852010 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 20582
I20260807 19:38:04.852195 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:38:04.852790 14401 external_mini_cluster.cc:1428] Setting key 691f18b8459ec93e915ef2f72cde9e7f
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:38:04.871104 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:04.854535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:59505 (local address 127.14.16.126:43653)
0807 19:38:04.854742 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:04.854748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:04.856183 (+ 1435us) server_negotiation.cc:407] Connection header received
0807 19:38:04.858044 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:04.858048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:04.858112 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:04.858184 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:04.859576 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.860316 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.861802 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.862203 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.862341 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:04.865605 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:04.865611 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:04.865621 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:04.865652 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:04.868637 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:04.869263 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:04.869267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:04.869269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:04.869393 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:04.869753 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:04.869756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:04.869770 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:04.870016 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:04.870124 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:04.870701 (+ 577us) server_negotiation.cc:299] Negotiation successful
0807 19:38:04.870865 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:38:04.871965 20717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:04.855124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:43653 (local address 127.14.16.67:59505)
0807 19:38:04.855996 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:04.856035 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:04.857809 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:04.858343 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:04.858350 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:04.858842 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:04.859429 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:04.859448 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.860454 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.860458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:04.861687 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:04.861700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.862435 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.862442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:04.862666 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:04.863271 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:04.863336 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:04.865409 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:04.868759 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:04.868766 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:04.868775 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:04.869159 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:04.869480 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:04.869483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:04.869485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:04.869652 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:04.870113 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:04.870119 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:04.870388 (+ 269us) client_negotiation.cc:769] Sending connection context
0807 19:38:04.870604 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 19:38:04.870946 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":381,"threads_started":1}
I20260807 19:38:04.873138 20715 heartbeater.cc:347] Connected to a master server at 127.14.16.126:43653
I20260807 19:38:04.873383 20715 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:04.874003 20715 heartbeater.cc:511] Master 127.14.16.126:43653 requested a full tablet report, sending...
I20260807 19:38:04.875247 20245 ts_manager.cc:194] Registered new tserver with Master: 15040874938a452ea604d1f1dbbf07a6 (127.14.16.67:44173)
I20260807 19:38:04.876057 20245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:59505
I20260807 19:38:04.880002 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/test-admin.keytab.
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:04.909874 14401 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 19:38:04.914808 20722 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:04.912487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36622 (local address 127.14.16.65:34365)
0807 19:38:04.913093 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:04.913099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:04.913127 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:38:04.913244 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:04.913251 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:04.913468 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:04.913600 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:04.914402 (+ 802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36622: BlockingRecv error: recv got EOF from 127.0.0.1:36622 (error 108)
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":284,"threads_started":1}
W20260807 19:38:04.915001 14401 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/krb5cc)
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:04.916549 14401 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:04.917919 14401 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:38:04.932740 20722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:04.919425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.14.16.65:34365)
0807 19:38:04.919636 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:04.919642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:04.919884 (+ 242us) server_negotiation.cc:407] Connection header received
0807 19:38:04.920032 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:04.920035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:04.920090 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:04.920161 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:04.921106 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.922048 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.923210 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.923826 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.923923 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:04.927353 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:04.927364 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:04.927376 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:04.927405 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:04.929948 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:04.930599 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:04.930606 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:04.930608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:04.930729 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:04.931140 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:04.931145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:04.931148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:04.931556 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:04.931856 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:04.932198 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 19:38:04.932477 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 19:38:04.938366 20722 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:04.936617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36642 (local address 127.14.16.65:34365)
0807 19:38:04.936810 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:04.936815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:04.937023 (+ 208us) server_negotiation.cc:407] Connection header received
0807 19:38:04.937197 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:04.937202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:04.937273 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:04.937367 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:04.938116 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36642: BlockingRecv error: recv got EOF from 127.0.0.1:36642 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 19:38:04.938575 14401 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/krb5cc' of type FILE
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:04.940050 14401 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:04.941350 14401 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:38:04.954773 20722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:04.942469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.14.16.65:34365)
0807 19:38:04.942659 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:04.942667 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:04.942949 (+ 282us) server_negotiation.cc:407] Connection header received
0807 19:38:04.943119 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:04.943124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:04.943184 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:04.943308 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:04.944189 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.944871 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.946061 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.946481 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.946607 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:04.949990 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:04.949996 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:04.950001 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:04.950048 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:04.952349 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:04.952986 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:04.952992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:04.952996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:04.953150 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:04.953529 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:04.953534 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:04.953538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:04.953795 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:04.954059 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:04.954326 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 19:38:04.954517 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 19:38:04.958947 20722 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:04.957220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.14.16.65:34365)
0807 19:38:04.957414 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:04.957421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:04.957681 (+ 260us) server_negotiation.cc:407] Connection header received
0807 19:38:04.957857 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:04.957862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:04.957932 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:04.958025 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:04.958720 (+ 695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36666: BlockingRecv error: recv got EOF from 127.0.0.1:36666 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 19:38:04.959089 14401 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131435615921-14401-0/krb5kdc/krb5cc' of type FILE
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:04.960531 14401 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:04.961790 14401 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 19:38:04 dist-test-slave-07c2 krb5kdc[20198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:38:04.975201 20722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:04.962886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.14.16.65:34365)
0807 19:38:04.963075 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:04.963084 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:04.963420 (+ 336us) server_negotiation.cc:407] Connection header received
0807 19:38:04.963596 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:04.963600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:04.963647 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:04.963737 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:04.964645 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.965325 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.966499 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.966939 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.967073 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:04.970367 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:04.970372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:04.970375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:04.970403 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:04.972730 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:04.973351 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:04.973357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:04.973361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:04.973495 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:04.973861 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:04.973867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:04.973870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:04.974135 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:04.974448 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:04.974685 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 19:38:04.974892 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:38:04.976994 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 20308
I20260807 19:38:04.988986 20722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:04.977697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36686 (local address 127.14.16.65:34365)
0807 19:38:04.977847 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:04.977851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:04.978152 (+ 301us) server_negotiation.cc:407] Connection header received
0807 19:38:04.978276 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:04.978279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:04.978327 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:04.978456 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:04.979526 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.980229 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.981470 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:04.981853 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:04.981949 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:04.984086 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:04.984091 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:04.984094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:04.984123 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:04.986448 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:04.987064 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:04.987069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:04.987073 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:04.987202 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:04.987599 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:04.987604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:04.987607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:04.987860 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:04.988141 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:04.988468 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 19:38:04.988680 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 19:38:04.989559 20435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:05.072163 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 20308
I20260807 19:38:05.091828 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 20446
Aug 07 19:38:05 dist-test-slave-07c2 krb5kdc[20198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:38:05.107218 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:05.092699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46768 (local address 127.14.16.66:44697)
0807 19:38:05.093262 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:05.093275 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:05.093312 (+ 37us) server_negotiation.cc:407] Connection header received
0807 19:38:05.093374 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:05.093379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:05.093624 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:05.093787 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:05.094863 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:05.095958 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:05.097269 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:05.097799 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:05.097932 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:05.101685 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:05.101695 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:05.101708 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:05.101748 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:05.104438 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:05.105125 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:05.105132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:05.105134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:05.105268 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:05.105632 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:05.105638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:05.105641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:05.105989 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:05.106332 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:05.106642 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 19:38:05.106908 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":259,"threads_started":1}
I20260807 19:38:05.108513 20571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:05.190832 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 20446
I20260807 19:38:05.212657 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 20582
Aug 07 19:38:05 dist-test-slave-07c2 krb5kdc[20198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:38:05.228077 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:05.213628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41752 (local address 127.14.16.67:44173)
0807 19:38:05.214277 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:05.214284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:05.214312 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:38:05.214375 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:05.214380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:05.214591 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:05.214773 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:05.216038 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:05.217063 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:05.218371 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:05.218939 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:05.219078 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:05.223000 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:05.223011 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:05.223024 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:05.223057 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:05.225744 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:05.226354 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:05.226360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:05.226362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:05.226481 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:05.226783 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:05.226786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:05.226788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:05.227108 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:05.227396 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:05.227579 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 19:38:05.227789 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":338,"threads_started":1}
I20260807 19:38:05.229343 20710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:05.314353 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 20582
I20260807 19:38:05.332834 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 20211
I20260807 19:38:05.333999 20275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:05.438671 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 20211
2026-08-07T19:38:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4040 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:38:05 dist-test-slave-07c2 krb5kdc[20742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:38:05 dist-test-slave-07c2 krb5kdc[20742](info): set up 2 sockets
Aug 07 19:38:05 dist-test-slave-07c2 krb5kdc[20742](info): commencing operation
krb5kdc: starting...
W20260807 19:38:06.559113 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:38:06 dist-test-slave-07c2 krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:38:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:38:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.126.keytab.
Entry for principal oryx/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.126.keytab.
I20260807 19:38:06.786958 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:41075
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.126.keytab
--principal=oryx/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:41075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:07.111027 20755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:07.111420 20755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:07.111577 20755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:07.118878 20755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:38:07.118984 20755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:07.119095 20755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:07.119187 20755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:38:07.119261 20755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:38:07.129251 20755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:41075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.126.keytab
--principal=oryx/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:41075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:07.130595 20755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:07.131947 20755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:07.143867 20761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:07.143736 20764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:07.143822 20762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:07.144716 20755 server_base.cc:1034] running on GCE node
I20260807 19:38:07.145419 20755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:07.146673 20755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:07.147874 20755 hybrid_clock.cc:648] HybridClock initialized: now 1786131487147823 us; error 34 us; skew 500 ppm
I20260807 19:38:07.148226 20755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:07 dist-test-slave-07c2 krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131487, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:07.151410 20755 init.cc:377] Logged in from keytab as oryx/127.14.16.126@KRBTEST.COM (short username oryx)
I20260807 19:38:07.152931 20755 webserver.cc:460] Webserver started at http://127.14.16.126:35613/ using document root <none> and password file <none>
I20260807 19:38:07.153621 20755 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:07.153769 20755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:07.154071 20755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:07.157091 20755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "128d4df4ad934462bd6508db5f4fe86f"
format_stamp: "Formatted at 2026-08-07 19:38:07 on dist-test-slave-07c2"
server_key: "8561cec30978dd3a40dad58d235de77d"
server_key_iv: "dbe3e4212e38aeb6c79445c1844b63f9"
server_key_version: "encryptionkey@0"
I20260807 19:38:07.157882 20755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "128d4df4ad934462bd6508db5f4fe86f"
format_stamp: "Formatted at 2026-08-07 19:38:07 on dist-test-slave-07c2"
server_key: "8561cec30978dd3a40dad58d235de77d"
server_key_iv: "dbe3e4212e38aeb6c79445c1844b63f9"
server_key_version: "encryptionkey@0"
I20260807 19:38:07.162894 20755 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:38:07.166218 20771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:07.167557 20755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:38:07.167730 20755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "128d4df4ad934462bd6508db5f4fe86f"
format_stamp: "Formatted at 2026-08-07 19:38:07 on dist-test-slave-07c2"
server_key: "8561cec30978dd3a40dad58d235de77d"
server_key_iv: "dbe3e4212e38aeb6c79445c1844b63f9"
server_key_version: "encryptionkey@0"
I20260807 19:38:07.167923 20755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:07.192342 20755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:07.196408 20755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:07.196705 20755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:07.216220 20755 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:41075
I20260807 19:38:07.216274 20826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:41075 every 8 connection(s)
I20260807 19:38:07.217727 20755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:38:07.218923 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 20755
I20260807 19:38:07.219151 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:38:07.220037 14401 external_mini_cluster.cc:1428] Setting key af4be4e92352f7106af0ffa70977cd57
I20260807 19:38:07.222750 20827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:38:07 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:07.235888 20827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f: Bootstrap starting.
I20260807 19:38:07.240001 20827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:07.241272 20827 log.cc:824] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:07.244210 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:07.222290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38990 (local address 127.14.16.126:41075)
0807 19:38:07.223127 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:07.223138 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:07.223179 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:38:07.224869 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:07.224889 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:07.225264 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:07.225592 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:07.227495 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:07.228836 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:07.230531 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:07.231173 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:07.231354 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:07.236035 (+ 4681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:07.236045 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:07.236063 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:07.236104 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:07.240024 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:07.240881 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:07.240890 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:07.240898 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:07.241092 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:07.241476 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:07.241481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:07.241485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:07.242232 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:07.242582 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:07.242870 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 19:38:07.243194 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":402,"threads_started":1}
I20260807 19:38:07.244457 20827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f: No bootstrap required, opened a new log
I20260807 19:38:07.251677 20827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } }
I20260807 19:38:07.252044 20827 raft_consensus.cc:374] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:07.252147 20827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128d4df4ad934462bd6508db5f4fe86f, State: Initialized, Role: FOLLOWER
I20260807 19:38:07.252751 20827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } }
I20260807 19:38:07.253028 20827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:38:07.253197 20827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:38:07.253367 20827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:07.255546 20827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } }
I20260807 19:38:07.256033 20827 leader_election.cc:302] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 128d4df4ad934462bd6508db5f4fe86f; no voters:
I20260807 19:38:07.256609 20827 leader_election.cc:288] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:38:07.256747 20832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:07.257439 20832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 1 LEADER]: Becoming Leader. State: Replica: 128d4df4ad934462bd6508db5f4fe86f, State: Running, Role: LEADER
I20260807 19:38:07.257993 20832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } }
I20260807 19:38:07.258709 20827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:38:07.260486 20834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 128d4df4ad934462bd6508db5f4fe86f. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } }
I20260807 19:38:07.260852 20834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:07.260883 20833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } }
I20260807 19:38:07.261441 20833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:07.270846 20837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:38:07.276629 20837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:38:07.285857 20837 catalog_manager.cc:1347] Generated new cluster ID: 542a815e91c4463a8da4ecf60e276a25
I20260807 19:38:07.285979 20837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:38:07.302134 20837 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:38:07.303423 20837 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:38:07.312299 20837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f: Generated new TSK 0
I20260807 19:38:07.313383 20837 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab.
Entry for principal oryx/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab.
I20260807 19:38:07.378427 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41075
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab
--principal=oryx/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:07.717077 20855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:07.717401 20855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:07.717576 20855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:07.724838 20855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:07.724989 20855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:07.725170 20855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:38:07.734923 20855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab
--principal=oryx/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:07.736280 20855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:07.737602 20855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:07.750892 20861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:07.750938 20860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:07.751157 20864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:07.751875 20855 server_base.cc:1034] running on GCE node
I20260807 19:38:07.752472 20855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:07.753405 20855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:07.754623 20855 hybrid_clock.cc:648] HybridClock initialized: now 1786131487754568 us; error 50 us; skew 500 ppm
I20260807 19:38:07.754953 20855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:07 dist-test-slave-07c2 krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131487, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:07.758112 20855 init.cc:377] Logged in from keytab as oryx/127.14.16.65@KRBTEST.COM (short username oryx)
I20260807 19:38:07.759662 20855 webserver.cc:460] Webserver started at http://127.14.16.65:33889/ using document root <none> and password file <none>
I20260807 19:38:07.760381 20855 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:07.760525 20855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:07.760828 20855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:07.763864 20855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "adcee9bcb55c4940b2cbce43184628df"
format_stamp: "Formatted at 2026-08-07 19:38:07 on dist-test-slave-07c2"
server_key: "d73747f6ee427cd6a2b54a9bdff18bae"
server_key_iv: "b782bb3be23f8170effa4f17255e2476"
server_key_version: "encryptionkey@0"
I20260807 19:38:07.764647 20855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "adcee9bcb55c4940b2cbce43184628df"
format_stamp: "Formatted at 2026-08-07 19:38:07 on dist-test-slave-07c2"
server_key: "d73747f6ee427cd6a2b54a9bdff18bae"
server_key_iv: "b782bb3be23f8170effa4f17255e2476"
server_key_version: "encryptionkey@0"
I20260807 19:38:07.769981 20855 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:38:07.773522 20871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:07.774880 20855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:38:07.775053 20855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "adcee9bcb55c4940b2cbce43184628df"
format_stamp: "Formatted at 2026-08-07 19:38:07 on dist-test-slave-07c2"
server_key: "d73747f6ee427cd6a2b54a9bdff18bae"
server_key_iv: "b782bb3be23f8170effa4f17255e2476"
server_key_version: "encryptionkey@0"
I20260807 19:38:07.775245 20855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:07.796912 20855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:07.800904 20855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:07.801182 20855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:07.802156 20855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:07.803517 20855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:07.803617 20855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:07.803773 20855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:07.803841 20855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:07.836863 20855 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:38289
I20260807 19:38:07.836900 20988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:38289 every 8 connection(s)
I20260807 19:38:07.838218 20855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:38:07.843155 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 20855
I20260807 19:38:07.843386 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:38:07.844031 14401 external_mini_cluster.cc:1428] Setting key fd1d6ddcc46856fc889f60b1f5dba184
Aug 07 19:38:07 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131487, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:07.858611 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:07.840752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:49661 (local address 127.14.16.126:41075)
0807 19:38:07.841204 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:07.841208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:07.842017 (+ 809us) server_negotiation.cc:407] Connection header received
0807 19:38:07.843974 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:07.843981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:07.844088 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:07.844218 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:07.845877 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:07.846547 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:07.848155 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:07.848572 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:07.848712 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:07.852600 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:07.852604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:07.852607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:07.852657 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:07.855431 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:07.856126 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:07.856133 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:07.856137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:07.856284 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:07.856685 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:07.856693 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:07.856699 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:07.857081 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:07.857244 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:07.857870 (+ 626us) server_negotiation.cc:299] Negotiation successful
0807 19:38:07.858134 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":241,"threads_started":1}
I20260807 19:38:07.859340 20992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:07.841152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.65:49661)
0807 19:38:07.841840 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:07.841884 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:07.843713 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:07.844443 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:07.844451 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:07.845068 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:07.845723 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:07.845743 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:07.846721 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:07.846726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:07.848001 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:07.848013 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:07.848801 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:07.848808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:07.849031 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:07.849710 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:07.849731 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:07.852386 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:07.855558 (+ 3172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:07.855565 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:07.855576 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:07.855998 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:07.856388 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:07.856391 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:07.856393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:07.856568 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:07.857200 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:07.857206 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:07.857499 (+ 293us) client_negotiation.cc:769] Sending connection context
0807 19:38:07.857713 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 19:38:07.858089 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":267,"threads_started":1}
I20260807 19:38:07.860997 20989 heartbeater.cc:347] Connected to a master server at 127.14.16.126:41075
I20260807 19:38:07.861308 20989 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:07.862001 20989 heartbeater.cc:511] Master 127.14.16.126:41075 requested a full tablet report, sending...
I20260807 19:38:07.864060 20790 ts_manager.cc:194] Registered new tserver with Master: adcee9bcb55c4940b2cbce43184628df (127.14.16.65:38289)
I20260807 19:38:07.865626 20790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:49661
WARNING: no policy specified for oryx/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab.
Entry for principal oryx/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab.
I20260807 19:38:07.898380 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41075
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab
--principal=oryx/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:08.228173 20995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:08.228473 20995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:08.228645 20995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:08.235754 20995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:08.235894 20995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:08.236075 20995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:38:08.246726 20995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab
--principal=oryx/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:08.248019 20995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:08.249337 20995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:08.262550 21000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:08.262519 21001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:08.262919 21003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:08.263430 20995 server_base.cc:1034] running on GCE node
I20260807 19:38:08.264035 20995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:08.264972 20995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:08.266165 20995 hybrid_clock.cc:648] HybridClock initialized: now 1786131488266105 us; error 51 us; skew 500 ppm
I20260807 19:38:08.266474 20995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:08 dist-test-slave-07c2 krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131488, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:08.269591 20995 init.cc:377] Logged in from keytab as oryx/127.14.16.66@KRBTEST.COM (short username oryx)
I20260807 19:38:08.271155 20995 webserver.cc:460] Webserver started at http://127.14.16.66:34897/ using document root <none> and password file <none>
I20260807 19:38:08.271937 20995 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:08.272081 20995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:08.272390 20995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:08.275561 20995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "bf7ee3db77e843c982f7c410345536f8"
format_stamp: "Formatted at 2026-08-07 19:38:08 on dist-test-slave-07c2"
server_key: "cb63eea0b0d0f06c29ca32f34d3c9e8c"
server_key_iv: "a8531d37e626134acec850415887705e"
server_key_version: "encryptionkey@0"
I20260807 19:38:08.276372 20995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "bf7ee3db77e843c982f7c410345536f8"
format_stamp: "Formatted at 2026-08-07 19:38:08 on dist-test-slave-07c2"
server_key: "cb63eea0b0d0f06c29ca32f34d3c9e8c"
server_key_iv: "a8531d37e626134acec850415887705e"
server_key_version: "encryptionkey@0"
I20260807 19:38:08.282090 20995 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:38:08.285800 21010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:08.287036 20995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:38:08.287210 20995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "bf7ee3db77e843c982f7c410345536f8"
format_stamp: "Formatted at 2026-08-07 19:38:08 on dist-test-slave-07c2"
server_key: "cb63eea0b0d0f06c29ca32f34d3c9e8c"
server_key_iv: "a8531d37e626134acec850415887705e"
server_key_version: "encryptionkey@0"
I20260807 19:38:08.287479 20995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:08.304621 20995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:08.308324 20995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:08.308607 20995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:08.309489 20995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:08.310850 20995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:08.310947 20995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:08.311086 20995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:08.311154 20995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:08.344157 20995 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:35213
I20260807 19:38:08.344203 21125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:35213 every 8 connection(s)
I20260807 19:38:08.345414 20995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:38:08.351605 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 20995
I20260807 19:38:08.351787 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:38:08.352218 14401 external_mini_cluster.cc:1428] Setting key e149c48a9afada4603e018d96716b4a6
Aug 07 19:38:08 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131488, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:08.366214 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:08.347940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:60775 (local address 127.14.16.126:41075)
0807 19:38:08.348143 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:08.348150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:08.349299 (+ 1149us) server_negotiation.cc:407] Connection header received
0807 19:38:08.350999 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:08.351004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:08.351071 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:08.351157 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:08.352774 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:08.353491 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:08.355259 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:08.356083 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:08.356229 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:08.359580 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:08.359588 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:08.359596 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:08.359640 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:08.363222 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:08.364027 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:08.364032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:08.364034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:08.364152 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:08.364610 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:08.364612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:08.364614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:08.364866 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:08.364973 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:08.365813 (+ 840us) server_negotiation.cc:299] Negotiation successful
0807 19:38:08.365975 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:38:08.367707 21128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:08.348447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.66:60775)
0807 19:38:08.349150 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:08.349193 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:08.350747 (+ 1554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:08.351411 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:08.351419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:08.352002 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:08.352600 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:08.352618 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:08.353652 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:08.353657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:08.355114 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:08.355126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:08.356285 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:08.356290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:08.356470 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:08.357112 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:08.357137 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:08.359357 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:08.363376 (+ 4019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:08.363382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:08.363392 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:08.363914 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:08.364255 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:08.364268 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:08.364272 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:08.364509 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:08.364990 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:08.364999 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:08.365429 (+ 430us) client_negotiation.cc:769] Sending connection context
0807 19:38:08.365723 (+ 294us) client_negotiation.cc:240] Negotiation successful
0807 19:38:08.366291 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":273,"threads_started":1}
I20260807 19:38:08.369014 21126 heartbeater.cc:347] Connected to a master server at 127.14.16.126:41075
I20260807 19:38:08.369297 21126 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:08.369999 21126 heartbeater.cc:511] Master 127.14.16.126:41075 requested a full tablet report, sending...
I20260807 19:38:08.371769 20790 ts_manager.cc:194] Registered new tserver with Master: bf7ee3db77e843c982f7c410345536f8 (127.14.16.66:35213)
I20260807 19:38:08.373070 20790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:60775
WARNING: no policy specified for oryx/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab.
Entry for principal oryx/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab.
I20260807 19:38:08.405664 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41075
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab
--principal=oryx/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:08.732146 21131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:08.732398 21131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:08.732494 21131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:08.739455 21131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:08.739575 21131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:08.739694 21131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:38:08.748888 21131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab
--principal=oryx/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:08.749977 21131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:08.751164 21131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:08.764348 21136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:08.764405 21137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:08.764441 21139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:08.765290 21131 server_base.cc:1034] running on GCE node
I20260807 19:38:08.765853 21131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:08.766705 21131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:08.767938 21131 hybrid_clock.cc:648] HybridClock initialized: now 1786131488767897 us; error 43 us; skew 500 ppm
I20260807 19:38:08.768266 21131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:08 dist-test-slave-07c2 krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131488, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:08.771351 21131 init.cc:377] Logged in from keytab as oryx/127.14.16.67@KRBTEST.COM (short username oryx)
I20260807 19:38:08.772692 21131 webserver.cc:460] Webserver started at http://127.14.16.67:39627/ using document root <none> and password file <none>
I20260807 19:38:08.773326 21131 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:08.773463 21131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:08.773759 21131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:08.776633 21131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "b170bb276e7c4f1cbe39f4c8208f7e69"
format_stamp: "Formatted at 2026-08-07 19:38:08 on dist-test-slave-07c2"
server_key: "058ed042b000c34c78db312a61de9248"
server_key_iv: "49cebe84ccca878f52020da0681df78c"
server_key_version: "encryptionkey@0"
I20260807 19:38:08.777388 21131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "b170bb276e7c4f1cbe39f4c8208f7e69"
format_stamp: "Formatted at 2026-08-07 19:38:08 on dist-test-slave-07c2"
server_key: "058ed042b000c34c78db312a61de9248"
server_key_iv: "49cebe84ccca878f52020da0681df78c"
server_key_version: "encryptionkey@0"
I20260807 19:38:08.782286 21131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 19:38:08.785614 21146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:08.786909 21131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:38:08.787032 21131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "b170bb276e7c4f1cbe39f4c8208f7e69"
format_stamp: "Formatted at 2026-08-07 19:38:08 on dist-test-slave-07c2"
server_key: "058ed042b000c34c78db312a61de9248"
server_key_iv: "49cebe84ccca878f52020da0681df78c"
server_key_version: "encryptionkey@0"
I20260807 19:38:08.787191 21131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:08.818656 21131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:08.822728 21131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:08.822988 21131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:08.823976 21131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:08.825317 21131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:08.825393 21131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:08.825486 21131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:08.825536 21131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:08.859382 21131 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:37913
I20260807 19:38:08.859467 21261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:37913 every 8 connection(s)
I20260807 19:38:08.860620 21131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:38:08.867172 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 21131
I20260807 19:38:08.867376 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:38:08.867854 14401 external_mini_cluster.cc:1428] Setting key 2fa4fa689a2ae96652f11b004bf4b862
I20260807 19:38:08.868563 20989 heartbeater.cc:503] Master 127.14.16.126:41075 was elected leader, sending a full tablet report...
Aug 07 19:38:08 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131488, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:08.880095 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:08.863061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:42759 (local address 127.14.16.126:41075)
0807 19:38:08.863244 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:08.863249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:08.864402 (+ 1153us) server_negotiation.cc:407] Connection header received
0807 19:38:08.866184 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:08.866191 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:08.866275 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:08.866385 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:08.867909 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:08.868937 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:08.870868 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:08.871400 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:08.871496 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:08.874767 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:08.874771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:08.874775 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:08.874804 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:08.877376 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:08.877992 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:08.877996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:08.877998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:08.878110 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:08.878551 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:08.878554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:08.878556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:08.878826 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:08.878936 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:08.879638 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 19:38:08.879794 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:38:08.881034 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:08.863565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.67:42759)
0807 19:38:08.864260 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:08.864296 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:08.865908 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:08.866607 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:08.866615 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:08.867090 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:08.867749 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:08.867768 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:08.869108 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:08.869113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:08.870752 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:08.870766 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:08.871580 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:08.871585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:08.871830 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:08.872444 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:08.872466 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:08.874608 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:08.877488 (+ 2880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:08.877494 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:08.877504 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:08.877894 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:08.878215 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:08.878218 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:08.878220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:08.878437 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:08.878940 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:08.878945 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:08.879230 (+ 285us) client_negotiation.cc:769] Sending connection context
0807 19:38:08.879564 (+ 334us) client_negotiation.cc:240] Negotiation successful
0807 19:38:08.880010 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":286,"threads_started":1}
I20260807 19:38:08.882712 21262 heartbeater.cc:347] Connected to a master server at 127.14.16.126:41075
I20260807 19:38:08.882988 21262 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:08.883809 21262 heartbeater.cc:511] Master 127.14.16.126:41075 requested a full tablet report, sending...
I20260807 19:38:08.885138 20790 ts_manager.cc:194] Registered new tserver with Master: b170bb276e7c4f1cbe39f4c8208f7e69 (127.14.16.67:37913)
I20260807 19:38:08.885928 20790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:42759
I20260807 19:38:08.895115 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.16.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.125.keytab.
Entry for principal oryx/127.14.16.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.125.keytab.
I20260807 19:38:08.951185 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.125:38459
--webserver_interface=127.14.16.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.125.keytab
--principal=oryx/127.14.16.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:41075,127.14.16.125:38459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:09.279935 21267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:09.280275 21267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:09.280433 21267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:09.287588 21267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:38:09.287734 21267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:09.287834 21267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:09.287933 21267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:38:09.288012 21267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:38:09.298231 21267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:41075,127.14.16.125:38459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.125.keytab
--principal=oryx/127.14.16.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.125:38459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.16.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
W20260807 19:38:09.298607 21267 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.14.16.126:41075,127.14.16.125:38459'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 19:38:09.299791 21267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:09.301079 21267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:09.312832 21272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:09.312911 21275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:09.312863 21273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:09.313843 21267 server_base.cc:1034] running on GCE node
I20260807 19:38:09.314584 21267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:09.315930 21267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:09.317145 21267 hybrid_clock.cc:648] HybridClock initialized: now 1786131489317078 us; error 40 us; skew 500 ppm
I20260807 19:38:09.317432 21267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:09 dist-test-slave-07c2 krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131489, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:09.320501 21267 init.cc:377] Logged in from keytab as oryx/127.14.16.125@KRBTEST.COM (short username oryx)
I20260807 19:38:09.321928 21267 webserver.cc:460] Webserver started at http://127.14.16.125:39817/ using document root <none> and password file <none>
I20260807 19:38:09.322592 21267 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:09.322706 21267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:09.323040 21267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:09.326032 21267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data/instance:
uuid: "00cff960e6a84647bf857c9b54f4c50a"
format_stamp: "Formatted at 2026-08-07 19:38:09 on dist-test-slave-07c2"
server_key: "1ae50388315894371b4b5d02f85b8d00"
server_key_iv: "e21acdf8bc0e070fd213f9668d036f26"
server_key_version: "encryptionkey@0"
I20260807 19:38:09.326799 21267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/wal/instance:
uuid: "00cff960e6a84647bf857c9b54f4c50a"
format_stamp: "Formatted at 2026-08-07 19:38:09 on dist-test-slave-07c2"
server_key: "1ae50388315894371b4b5d02f85b8d00"
server_key_iv: "e21acdf8bc0e070fd213f9668d036f26"
server_key_version: "encryptionkey@0"
I20260807 19:38:09.332093 21267 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:38:09.335493 21282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:09.336714 21267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:38:09.336884 21267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/wal
uuid: "00cff960e6a84647bf857c9b54f4c50a"
format_stamp: "Formatted at 2026-08-07 19:38:09 on dist-test-slave-07c2"
server_key: "1ae50388315894371b4b5d02f85b8d00"
server_key_iv: "e21acdf8bc0e070fd213f9668d036f26"
server_key_version: "encryptionkey@0"
I20260807 19:38:09.337083 21267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:09.352311 21267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:09.356396 21267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:09.356665 21267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:09.375887 21126 heartbeater.cc:503] Master 127.14.16.126:41075 was elected leader, sending a full tablet report...
I20260807 19:38:09.375872 21267 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.125:38459
I20260807 19:38:09.375869 21334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.125:38459 every 8 connection(s)
I20260807 19:38:09.378036 21267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
I20260807 19:38:09.381906 21335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:09.384938 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 21267
I20260807 19:38:09.390796 21335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:38:09 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.16.125@KRBTEST.COM
Aug 07 19:38:09 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131489, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.125@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:09.410068 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.386951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56822 (local address 127.14.16.125:38459)
0807 19:38:09.388611 (+ 1660us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:09.388624 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:09.388663 (+ 39us) server_negotiation.cc:407] Connection header received
0807 19:38:09.390308 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:09.390379 (+ 71us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:09.390879 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:09.391382 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:09.393212 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.394886 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.396448 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.397516 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.397767 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.402777 (+ 5010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:09.402789 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:09.402810 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:09.402841 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:09.405763 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.406457 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.406466 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.406475 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.406670 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.407076 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.407084 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.407088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.407814 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:09.408054 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:09.408634 (+ 580us) server_negotiation.cc:299] Negotiation successful
0807 19:38:09.409076 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1440,"thread_start_us":1176,"threads_started":1}
I20260807 19:38:09.410485 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.393894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54260 (local address 127.14.16.126:41075)
0807 19:38:09.394429 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:09.394437 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:09.395198 (+ 761us) server_negotiation.cc:407] Connection header received
0807 19:38:09.395450 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:09.395456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:09.395536 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:09.395648 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:09.396739 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.398015 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.400101 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.400516 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.400615 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.404284 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:09.404289 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:09.404292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:09.404348 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:09.407444 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.408312 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.408319 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.408325 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.408492 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.408966 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.408970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.408972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.409295 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:09.409453 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:09.409930 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 19:38:09.410142 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":286,"threads_started":1}
I20260807 19:38:09.410576 21340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.393927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54260)
0807 19:38:09.395028 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:09.395079 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:09.395259 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:09.395725 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:09.395730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:09.396306 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:09.396578 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.396589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.398183 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.398188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.399934 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.399953 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.400749 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.400759 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.400960 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.401654 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:09.401675 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:09.404067 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:09.407590 (+ 3523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:09.407598 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:09.407604 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:09.408152 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:09.408644 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:09.408648 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:09.408651 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:09.408826 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:09.409442 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:09.409450 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:09.409618 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 19:38:09.409811 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 19:38:09.410133 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"thread_start_us":566,"threads_started":1}
I20260807 19:38:09.412452 21335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:38:09 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131489, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.125@KRBTEST.COM for oryx/127.14.16.125@KRBTEST.COM
I20260807 19:38:09.424360 21340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.413279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:38459 (local address 127.0.0.1:56826)
0807 19:38:09.413452 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:09.413467 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:09.413542 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:09.413802 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:09.413804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:09.414108 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:09.414237 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.414244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.415169 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.415172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.416078 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.416089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.416927 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.416931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.417083 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.417661 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:09.417677 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:09.419659 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:09.422318 (+ 2659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:09.422322 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:09.422324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:09.422667 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:09.422984 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:09.422987 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:09.422989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:09.423110 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:09.423636 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:09.423639 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:09.423768 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 19:38:09.423877 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 19:38:09.424087 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 19:38:09.424490 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.413363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56826 (local address 127.14.16.125:38459)
0807 19:38:09.413507 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:09.413511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:09.413526 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:38:09.413632 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:09.413636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:09.413686 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:09.413765 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:09.414362 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.415047 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.416198 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.416796 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.416902 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.419820 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:09.419825 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:09.419828 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:09.419856 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:09.422181 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.422767 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.422771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.422773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.422889 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.423201 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.423204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.423206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.423538 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:09.423648 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:09.423969 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 19:38:09.424196 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":61}
I20260807 19:38:09.429154 21335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: Bootstrap starting.
I20260807 19:38:09.432353 21335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:09.433413 21335 log.cc:824] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:09.436144 21335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: No bootstrap required, opened a new log
I20260807 19:38:09.443626 21335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } }
I20260807 19:38:09.443994 21335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:09.444121 21335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cff960e6a84647bf857c9b54f4c50a, State: Initialized, Role: FOLLOWER
I20260807 19:38:09.444751 21335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } }
I20260807 19:38:09.445617 21342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } } }
I20260807 19:38:09.445901 21342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:38:09.446630 21335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: configured and running, proceeding with master startup.
W20260807 19:38:09.455153 21353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:38:09.455346 21353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:38:09.458540 21267 master_runner.cc:428] Detected that this master 00cff960e6a84647bf857c9b54f4c50a is joining an existing cluster
I20260807 19:38:09.458648 21267 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.16.125:38459
I20260807 19:38:09.481408 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.462628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54268 (local address 127.14.16.126:41075)
0807 19:38:09.462837 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:09.462843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:09.463508 (+ 665us) server_negotiation.cc:407] Connection header received
0807 19:38:09.463807 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:09.463813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:09.463916 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:09.463998 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:09.465940 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.467428 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.470194 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.471033 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.471172 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.474196 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:09.474203 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:09.474208 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:09.474264 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:09.477992 (+ 3728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.478856 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.478862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.478866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.479027 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.479551 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.479557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.479560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.479970 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:09.480098 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:09.480771 (+ 673us) server_negotiation.cc:299] Negotiation successful
0807 19:38:09.481034 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:38:09.483227 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.462638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54268)
0807 19:38:09.463410 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:09.463430 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:09.463625 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:09.464236 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:09.464241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:09.465164 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:09.465703 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.465722 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.467662 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.467670 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.469965 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.469986 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.471558 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.471566 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.471747 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.472640 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:09.472666 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:09.473982 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:09.478160 (+ 4178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:09.478165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:09.478169 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:09.478677 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:09.479182 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:09.479187 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:09.479191 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:09.479411 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:09.480205 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:09.480210 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:09.480423 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 19:38:09.481683 (+ 1260us) client_negotiation.cc:240] Negotiation successful
0807 19:38:09.482045 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":405,"threads_started":1}
I20260807 19:38:09.490576 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.463631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56836 (local address 127.14.16.125:38459)
0807 19:38:09.463838 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:09.463844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:09.471199 (+ 7355us) server_negotiation.cc:407] Connection header received
0807 19:38:09.471640 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:09.471645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:09.471716 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:09.471779 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:09.473489 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.474701 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.476794 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.477503 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.477615 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.480996 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:09.481005 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:09.481012 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:09.481057 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:09.486349 (+ 5292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.487822 (+ 1473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.487830 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.487836 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.487979 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.488428 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.488435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.488441 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.488887 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:09.489041 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:09.489718 (+ 677us) server_negotiation.cc:299] Negotiation successful
0807 19:38:09.489997 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for oryx/127.14.16.124@KRBTEST.COM; defaulting to no policy
I20260807 19:38:09.498663 21360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.464464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:38459 (local address 127.0.0.1:56836)
0807 19:38:09.471109 (+ 6645us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:09.471130 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:09.471459 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:09.471973 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:09.471981 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:09.472925 (+ 944us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:09.473330 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.473345 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.474937 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.474944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.476645 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.476660 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.477928 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.477935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.478145 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.479461 (+ 1316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:09.479488 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:09.480792 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:09.486537 (+ 5745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:09.486544 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:09.486551 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:09.487691 (+ 1140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:09.488135 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:09.488142 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:09.488148 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:09.488319 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:09.489129 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:09.489136 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:09.489327 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 19:38:09.497111 (+ 7784us) client_negotiation.cc:240] Negotiation successful
0807 19:38:09.497655 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6501,"spinlock_wait_cycles":29056,"thread_start_us":6726,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.124@KRBTEST.COM" created.
Aug 07 19:38:09 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131487, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.126@KRBTEST.COM for oryx/127.14.16.125@KRBTEST.COM
I20260807 19:38:09.534891 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.491224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56844 (local address 127.14.16.125:38459)
0807 19:38:09.493267 (+ 2043us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:09.493273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:09.515942 (+ 22669us) server_negotiation.cc:407] Connection header received
0807 19:38:09.516435 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:09.516444 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:09.516546 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:09.516671 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:09.518433 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.519718 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.521908 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.522633 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.522780 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.527453 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:09.527463 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:09.527471 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:09.527524 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:09.531652 (+ 4128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.532662 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.532669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.532673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.532828 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.533319 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.533327 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.533330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.533630 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:09.533804 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:09.534335 (+ 531us) server_negotiation.cc:299] Negotiation successful
0807 19:38:09.534608 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1926,"thread_start_us":23719,"threads_started":1}
I20260807 19:38:09.535108 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.490782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:38459 (local address 127.0.0.1:56844)
0807 19:38:09.515739 (+ 24957us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:09.515803 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:09.516144 (+ 341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:09.516954 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:09.516962 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:09.517885 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:09.518215 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.518229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.519878 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.519885 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.521730 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.521748 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.522839 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.522847 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.523089 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.523946 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:09.523977 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:09.527217 (+ 3240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:09.531829 (+ 4612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:09.531842 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:09.531851 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:09.532473 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:09.532961 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:09.532967 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:09.532971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:09.533163 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:09.533773 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:09.533783 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:09.533994 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 19:38:09.534231 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 19:38:09.534595 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24637,"thread_start_us":24487,"threads_started":1}
I20260807 19:38:09.537348 20790 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.16.125:38459: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a"
member_type: NON_VOTER
last_known_addr {
host: "127.14.16.125"
port: 38459
}
attrs {
promote: true
}
}
dest_uuid: "128d4df4ad934462bd6508db5f4fe86f"
cas_config_opid_index: -1
I20260807 19:38:09.538491 20790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.124.keytab.
Entry for principal oryx/127.14.16.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.124.keytab.
I20260807 19:38:09.540899 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.124:44121
--webserver_interface=127.14.16.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.124.keytab
--principal=oryx/127.14.16.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:41075,127.14.16.125:38459,127.14.16.124:44121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:38:09.553232 21368 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 00cff960e6a84647bf857c9b54f4c50a (127.14.16.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } } }
I20260807 19:38:09.554322 21309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:09.557121 21309 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 FOLLOWER]: Refusing update from remote peer 128d4df4ad934462bd6508db5f4fe86f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 19:38:09.557432 21368 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.16.125:38459
I20260807 19:38:09.559985 21267 master.cc:537] Master@127.14.16.125:38459 shutting down...
I20260807 19:38:09.561486 21370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } } }
I20260807 19:38:09.567800 21370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:09.560194 21367 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.008s
W20260807 19:38:09.577541 20775 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 19:38:09.583192 21267 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:09.583675 21267 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:09.583881 21267 tablet_replica.cc:330] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: stopping tablet replica
W20260807 19:38:09.591634 20775 consensus_peers.cc:487] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f -> Peer 00cff960e6a84647bf857c9b54f4c50a (127.14.16.125:38459): Couldn't send request to peer 00cff960e6a84647bf857c9b54f4c50a. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:38:09.591742 21267 master.cc:552] Master@127.14.16.125:38459 shutdown complete.
I20260807 19:38:09.591872 21267 master_runner.cc:305] Clearing existing system tablet
I20260807 19:38:09.592332 21267 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:09.592430 21267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:09.595552 21267 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 19:38:09.597255 21374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:09.597718 21267 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:09.597990 21267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 19:38:09.598084 21267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/wal
uuid: "00cff960e6a84647bf857c9b54f4c50a"
format_stamp: "Formatted at 2026-08-07 19:38:09 on dist-test-slave-07c2"
server_key: "1ae50388315894371b4b5d02f85b8d00"
server_key_iv: "e21acdf8bc0e070fd213f9668d036f26"
server_key_version: "encryptionkey@0"
I20260807 19:38:09.602946 21267 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:09.610225 21267 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 19:38:09.610421 21267 log.cc:1196] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 19:38:09.610862 21267 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: Deleting consensus metadata
I20260807 19:38:09.611148 21267 master_runner.cc:315] Copying system tablet from 127.14.16.126:41075
I20260807 19:38:09.613722 21267 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: tablet copy: Beginning tablet copy session from remote peer at address 127.14.16.126:41075
I20260807 19:38:09.630368 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.614558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54274 (local address 127.14.16.126:41075)
0807 19:38:09.614753 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:09.614760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:09.615864 (+ 1104us) server_negotiation.cc:407] Connection header received
0807 19:38:09.616077 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:09.616086 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:09.616203 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:09.616333 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:09.617408 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.618721 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.620628 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.621354 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.621485 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.623758 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:09.623765 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:09.623770 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:09.623810 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:09.627360 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.628153 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.628159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.628162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.628345 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:09.628792 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:09.628798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:09.628804 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:09.629153 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:09.629313 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:09.629778 (+ 465us) server_negotiation.cc:299] Negotiation successful
0807 19:38:09.629997 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:38:09.630429 21380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.614572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54274)
0807 19:38:09.615718 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:09.615745 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:09.615903 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:09.616370 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:09.616375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:09.616921 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:09.617211 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.617223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.618895 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.618901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.620430 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.620444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.621522 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.621529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.621692 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.622419 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:09.622440 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:09.623540 (+ 1100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:09.627494 (+ 3954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:09.627499 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:09.627503 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:09.628001 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:09.628465 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:09.628470 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:09.628474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:09.628660 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:09.629298 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:09.629305 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:09.629491 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 19:38:09.629631 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 19:38:09.629966 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"thread_start_us":910,"threads_started":1}
I20260807 19:38:09.631698 20811 tablet_copy_service.cc:140] P 128d4df4ad934462bd6508db5f4fe86f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 00cff960e6a84647bf857c9b54f4c50a ({username='oryx', principal='oryx/127.14.16.125@KRBTEST.COM'} at 127.0.0.1:54274)
I20260807 19:38:09.631980 20811 tablet_copy_service.cc:161] P 128d4df4ad934462bd6508db5f4fe86f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 00cff960e6a84647bf857c9b54f4c50a at {username='oryx', principal='oryx/127.14.16.125@KRBTEST.COM'} at 127.0.0.1:54274: session id = 00cff960e6a84647bf857c9b54f4c50a-00000000000000000000000000000000
I20260807 19:38:09.634892 20811 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:38:09.637070 21267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 19:38:09.643914 21267 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: tablet copy: Starting download of 0 data blocks...
I20260807 19:38:09.644213 21267 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: tablet copy: Starting download of 1 WAL segments...
I20260807 19:38:09.646943 21267 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:38:09.651510 20811 tablet_copy_service.cc:342] P 128d4df4ad934462bd6508db5f4fe86f: Request end of tablet copy session 00cff960e6a84647bf857c9b54f4c50a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.16.125@KRBTEST.COM'} at 127.0.0.1:54274
I20260807 19:38:09.651718 20811 tablet_copy_service.cc:433] P 128d4df4ad934462bd6508db5f4fe86f: ending tablet copy session 00cff960e6a84647bf857c9b54f4c50a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 00cff960e6a84647bf857c9b54f4c50a
I20260807 19:38:09.663180 21267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:09.666018 21381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:09.668521 21382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:09.680518 21267 server_base.cc:1034] running on GCE node
W20260807 19:38:09.680609 21384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:09.681107 21267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:09.681892 21267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:09.683676 21267 hybrid_clock.cc:648] HybridClock initialized: now 1786131489683573 us; error 49 us; skew 500 ppm
I20260807 19:38:09.684027 21267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:09 dist-test-slave-07c2 krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131489, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:09.686677 21267 init.cc:377] Logged in from keytab as oryx/127.14.16.125@KRBTEST.COM (short username oryx)
I20260807 19:38:09.688221 21267 webserver.cc:460] Webserver started at http://127.14.16.125:40207/ using document root <none> and password file <none>
I20260807 19:38:09.688699 21267 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:09.688841 21267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:09.692530 21267 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 19:38:09.694645 21391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:09.695129 21267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260807 19:38:09.695360 21267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/wal
uuid: "00cff960e6a84647bf857c9b54f4c50a"
format_stamp: "Formatted at 2026-08-07 19:38:09 on dist-test-slave-07c2"
server_key: "1ae50388315894371b4b5d02f85b8d00"
server_key_iv: "e21acdf8bc0e070fd213f9668d036f26"
server_key_version: "encryptionkey@0"
I20260807 19:38:09.695602 21267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:09.717114 21267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:09.721741 21267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:09.778185 21267 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.125:38459
I20260807 19:38:09.778208 21443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.125:38459 every 8 connection(s)
I20260807 19:38:09.780097 21267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
I20260807 19:38:09.786010 21444 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:38:09.789996 21444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: Bootstrap starting.
I20260807 19:38:09.807861 21444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:38:09.808396 21444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: Bootstrap complete.
I20260807 19:38:09.809553 21444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } }
I20260807 19:38:09.809832 21444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 00cff960e6a84647bf857c9b54f4c50a, State: Initialized, Role: LEARNER
I20260807 19:38:09.810083 21444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } }
I20260807 19:38:09.810995 21447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } } }
I20260807 19:38:09.811439 21447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: This master's current role is: LEARNER
I20260807 19:38:09.811246 21444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:38:09.820364 21459 catalog_manager.cc:1259] Loaded cluster ID: 542a815e91c4463a8da4ecf60e276a25
I20260807 19:38:09.820472 21459 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: loading cluster ID for follower catalog manager: success
I20260807 19:38:09.824744 21459 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: acquiring CA information for follower catalog manager: success
I20260807 19:38:09.827630 21459 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:38:09.888288 21262 heartbeater.cc:503] Master 127.14.16.126:41075 was elected leader, sending a full tablet report...
W20260807 19:38:09.968927 21369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:09.969216 21369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:09.969307 21369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:09.977456 21369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:38:09.977562 21369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:09.977738 21369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:09.977830 21369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:38:09.977896 21369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:38:09.979717 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.973074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:38459 (local address 127.0.0.1:56854)
0807 19:38:09.973282 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:09.973299 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:09.973407 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:09.974292 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:09.974297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:09.974348 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:09.974561 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.974572 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.976085 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.976098 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.977340 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:09.977353 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.978817 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.978823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:09.979037 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.979110 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 19:38:09.979216 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 19:38:09.979382 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 19:38:09.979769 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:09.973308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56854 (local address 127.14.16.125:38459)
0807 19:38:09.973880 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:09.973885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:09.973900 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:38:09.973962 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:09.973966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:09.974032 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:09.974245 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:09.974753 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.975925 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.977502 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:09.978614 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:09.978786 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:09.979041 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:09.979254 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 19:38:09.979423 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":321,"threads_started":1}
I20260807 19:38:09.981072 21418 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260807 19:38:09.981587 21447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 128d4df4ad934462bd6508db5f4fe86f. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } } }
I20260807 19:38:09.981886 21447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: This master's current role is: LEARNER
I20260807 19:38:09.989060 21369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:41075,127.14.16.125:38459,127.14.16.124:44121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.124.keytab
--principal=oryx/127.14.16.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.124:44121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.16.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:09.990381 21369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:09.991940 21369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:10.003998 21465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:10.004065 21467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:10.004065 21469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:10.005040 21369 server_base.cc:1034] running on GCE node
I20260807 19:38:10.005769 21369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:10.007066 21369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:10.008265 21369 hybrid_clock.cc:648] HybridClock initialized: now 1786131490008190 us; error 61 us; skew 500 ppm
I20260807 19:38:10.008627 21369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131490, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:10.011837 21369 init.cc:377] Logged in from keytab as oryx/127.14.16.124@KRBTEST.COM (short username oryx)
I20260807 19:38:10.013470 21369 webserver.cc:460] Webserver started at http://127.14.16.124:46633/ using document root <none> and password file <none>
I20260807 19:38:10.014168 21369 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:10.014314 21369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:10.014575 21369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:10.017669 21369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data/instance:
uuid: "d00a05c1214f47fc8dfaafec40e4f231"
format_stamp: "Formatted at 2026-08-07 19:38:10 on dist-test-slave-07c2"
server_key: "ab91898ed567538bc86f27fea740f1f7"
server_key_iv: "fec1614199d1baa089c6fd96fc59abaf"
server_key_version: "encryptionkey@0"
I20260807 19:38:10.018383 21369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/wal/instance:
uuid: "d00a05c1214f47fc8dfaafec40e4f231"
format_stamp: "Formatted at 2026-08-07 19:38:10 on dist-test-slave-07c2"
server_key: "ab91898ed567538bc86f27fea740f1f7"
server_key_iv: "fec1614199d1baa089c6fd96fc59abaf"
server_key_version: "encryptionkey@0"
I20260807 19:38:10.023437 21369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 19:38:10.026613 21476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:10.028096 21369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:38:10.028280 21369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/wal
uuid: "d00a05c1214f47fc8dfaafec40e4f231"
format_stamp: "Formatted at 2026-08-07 19:38:10 on dist-test-slave-07c2"
server_key: "ab91898ed567538bc86f27fea740f1f7"
server_key_iv: "fec1614199d1baa089c6fd96fc59abaf"
server_key_version: "encryptionkey@0"
I20260807 19:38:10.028473 21369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:10.055371 21369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:10.059558 21369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:10.059893 21369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:10.079185 21369 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.124:44121
I20260807 19:38:10.079217 21527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.124:44121 every 8 connection(s)
I20260807 19:38:10.080636 21369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
I20260807 19:38:10.084414 21528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:10.089025 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 21369
I20260807 19:38:10.092661 21528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.16.124@KRBTEST.COM
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131490, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.124@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:10.111994 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.096679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.14.16.126:41075)
0807 19:38:10.096887 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.096893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.097421 (+ 528us) server_negotiation.cc:407] Connection header received
0807 19:38:10.097580 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.097583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.097646 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.097731 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.099194 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.099931 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.101396 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.102013 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.102155 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.105951 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.105956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.105959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.105991 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.108525 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.109272 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.109281 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.109288 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.109471 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.110007 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.110015 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.110021 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.110385 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.110503 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.111180 (+ 677us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.111478 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:38:10.113049 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.096516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54286)
0807 19:38:10.097262 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.097302 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:10.097461 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:10.097850 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:10.097857 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:10.098663 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:10.099035 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.099044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.100083 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.100090 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.101246 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.101258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.102261 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.102269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.102433 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.103043 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:10.103065 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:10.105761 (+ 2696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:10.108678 (+ 2917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.108691 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.108707 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.109140 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.109600 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.109603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.109605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.109852 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.110518 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:10.110526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:10.110738 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 19:38:10.111066 (+ 328us) client_negotiation.cc:240] Negotiation successful
0807 19:38:10.111708 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":131,"thread_start_us":285,"threads_started":1}
I20260807 19:38:10.113070 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.091409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41274 (local address 127.14.16.124:44121)
0807 19:38:10.092289 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.092301 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.092341 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:38:10.093831 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.093850 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.094227 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.094521 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.096593 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.098763 (+ 2170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.100187 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.100911 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.101130 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.105655 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.105667 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.105691 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.105732 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.108638 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.109347 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.109367 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.109373 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.109579 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.110043 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.110046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.110048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.110591 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.110933 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.111150 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.111582 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":634,"thread_start_us":397,"threads_started":1}
I20260807 19:38:10.115686 21528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131490, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.124@KRBTEST.COM for oryx/127.14.16.125@KRBTEST.COM
I20260807 19:38:10.127244 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.116392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:38459 (local address 127.0.0.1:56856)
0807 19:38:10.116605 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.116621 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:10.116699 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:10.116997 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:10.116999 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:10.117300 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:10.117434 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.117441 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.118367 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.118370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.119268 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.119278 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.119976 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.119980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.120124 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.120612 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:10.120628 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:10.122510 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:10.125317 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.125320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.125323 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.125673 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.125999 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.126001 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.126004 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.126127 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.126572 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:10.126574 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:10.126689 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 19:38:10.126810 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 19:38:10.127015 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 19:38:10.127350 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.116503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56856 (local address 127.14.16.125:38459)
0807 19:38:10.116651 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.116656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.116690 (+ 34us) server_negotiation.cc:407] Connection header received
0807 19:38:10.116820 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.116824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.116885 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.116964 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.117549 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.118248 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.119438 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.119847 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.119941 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.122688 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.122693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.122696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.122727 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.125199 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.125769 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.125773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.125775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.125904 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.126224 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.126228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.126230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.126477 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.126587 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.126898 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.127069 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 19:38:10.128402 21528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131490, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.124@KRBTEST.COM for oryx/127.14.16.124@KRBTEST.COM
I20260807 19:38:10.140355 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.129183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:44121 (local address 127.0.0.1:41290)
0807 19:38:10.129319 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.129338 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:10.129440 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:10.129723 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:10.129727 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:10.130082 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:10.130204 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.130212 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.131134 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.131141 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.132134 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.132146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.132812 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.132817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.133065 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.133624 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:10.133640 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:10.135584 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:10.138250 (+ 2666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.138253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.138256 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.138597 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.138958 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.138961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.138963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.139108 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.139619 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:10.139622 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:10.139749 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 19:38:10.139872 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 19:38:10.140047 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":6}
I20260807 19:38:10.140431 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.129127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.14.16.124:44121)
0807 19:38:10.129315 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.129321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.129387 (+ 66us) server_negotiation.cc:407] Connection header received
0807 19:38:10.129566 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.129569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.129617 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.129707 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.130342 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.131009 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.132282 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.132661 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.132806 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.135776 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.135781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.135784 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.135811 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.138133 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.138692 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.138695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.138697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.138832 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.139196 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.139199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.139201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.139521 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.139630 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.139960 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.140171 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":54}
I20260807 19:38:10.145290 21528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: Bootstrap starting.
I20260807 19:38:10.148340 21528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:10.149375 21528 log.cc:824] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:10.151944 21528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: No bootstrap required, opened a new log
I20260807 19:38:10.158789 21528 raft_consensus.cc:348] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } }
I20260807 19:38:10.159210 21528 raft_consensus.cc:374] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:10.159358 21528 raft_consensus.cc:729] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d00a05c1214f47fc8dfaafec40e4f231, State: Initialized, Role: FOLLOWER
I20260807 19:38:10.159965 21528 consensus_queue.cc:260] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } }
I20260807 19:38:10.160841 21535 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } } }
I20260807 19:38:10.161176 21535 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:38:10.161837 21528 sys_catalog.cc:564] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:38:10.167940 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 20855
W20260807 19:38:10.171271 21546 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:38:10.171433 21546 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:38:10.176136 21369 master_runner.cc:428] Detected that this master d00a05c1214f47fc8dfaafec40e4f231 is joining an existing cluster
I20260807 19:38:10.176295 21369 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.16.124:44121
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.16.65@KRBTEST.COM
I20260807 19:38:10.202926 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.182508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41296 (local address 127.14.16.124:44121)
0807 19:38:10.182676 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.182680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.183594 (+ 914us) server_negotiation.cc:407] Connection header received
0807 19:38:10.183793 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.183797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.183861 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.183955 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.184910 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.185599 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.187535 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.188235 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.188335 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.194087 (+ 5752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.194092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.194097 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.194135 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.199708 (+ 5573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.200546 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.200550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.200553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.200687 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.201131 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.201135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.201139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.201510 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.201677 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.202301 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.202525 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 19:38:10.203120 21548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.169064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54762 (local address 127.14.16.65:38289)
0807 19:38:10.179838 (+ 10774us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.179847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.179877 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:38:10.179949 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.179957 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.180196 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.183465 (+ 3269us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.183731 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.185754 (+ 2023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.187683 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.188640 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.189396 (+ 756us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.194159 (+ 4763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.194174 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.194196 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.194259 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.198683 (+ 4424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.199654 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.199663 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.199666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.199837 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.200350 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.200355 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.200359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.200819 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.202287 (+ 1468us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.202449 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.202709 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10582,"thread_start_us":10425,"threads_started":1}
I20260807 19:38:10.204641 20984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:10.211141 21554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.182640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:44121 (local address 127.0.0.1:41296)
0807 19:38:10.183491 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.183508 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:10.183662 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:10.183971 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:10.183975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:10.184499 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:10.184777 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.184786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.185787 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.185791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.187384 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.187400 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.188669 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.188678 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.188876 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.192596 (+ 3720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:10.192625 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:10.193900 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:10.199870 (+ 5970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.199878 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.199885 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.200432 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.200836 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.200840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.200843 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.201015 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.201763 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:10.201767 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:10.201978 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 19:38:10.203194 (+ 1216us) client_negotiation.cc:240] Negotiation successful
0807 19:38:10.210551 (+ 7357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":389,"threads_started":1}
I20260807 19:38:10.211920 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.183517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54288)
0807 19:38:10.193637 (+ 10120us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.193657 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:10.193794 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:10.194700 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:10.194705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:10.195905 (+ 1200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:10.196365 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.196377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.198289 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.198294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.200159 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.200174 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.201488 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.201496 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.201679 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.202341 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:10.202364 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:10.203494 (+ 1130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:10.208168 (+ 4674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.208176 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.208183 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.209004 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.209597 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.209605 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.209611 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.209828 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.210622 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:10.210627 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:10.210872 (+ 245us) client_negotiation.cc:769] Sending connection context
0807 19:38:10.211039 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 19:38:10.211442 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9958,"thread_start_us":9742,"threads_started":1}
I20260807 19:38:10.212881 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.183253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54288 (local address 127.14.16.126:41075)
0807 19:38:10.183890 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.183896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.194036 (+ 10140us) server_negotiation.cc:407] Connection header received
0807 19:38:10.194114 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.194122 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.194212 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.194327 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.196550 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.197715 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.200363 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.201065 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.201175 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.203703 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.203709 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.203714 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.203753 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.207946 (+ 4193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.209208 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.209217 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.209233 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.209436 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.209985 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.209992 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.209998 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.210392 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.210553 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.212275 (+ 1722us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.212517 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496}
I20260807 19:38:10.233469 21556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.195355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:38459 (local address 127.0.0.1:56870)
0807 19:38:10.206974 (+ 11619us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.206995 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:10.207353 (+ 358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:10.207901 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:10.207906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:10.208801 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:10.209192 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.209205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.210691 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.210696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.212643 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.212658 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.214662 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.214670 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.214891 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.224955 (+ 10064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:10.224990 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:10.226215 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:10.230305 (+ 4090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.230310 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.230314 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.230812 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.231330 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.231334 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.231337 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.231516 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.232242 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:10.232247 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:10.232471 (+ 224us) client_negotiation.cc:769] Sending connection context
0807 19:38:10.232652 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 19:38:10.232998 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11485,"thread_start_us":11746,"threads_started":1}
I20260807 19:38:10.234406 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.193927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56870 (local address 127.14.16.125:38459)
0807 19:38:10.194611 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.194618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.207081 (+ 12463us) server_negotiation.cc:407] Connection header received
0807 19:38:10.207519 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.207525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.207611 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.207685 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.209396 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.210501 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.212806 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.213480 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.213581 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.226468 (+ 12887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.226475 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.226479 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.226528 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.230127 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.230970 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.230977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.230982 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.231144 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.231663 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.231668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.231671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.232023 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.232164 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.233810 (+ 1646us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.234051 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533}
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131487, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.126@KRBTEST.COM for oryx/127.14.16.124@KRBTEST.COM
I20260807 19:38:10.245404 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.224208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:44121 (local address 127.0.0.1:41302)
0807 19:38:10.224400 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.224419 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:10.224520 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:10.225548 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:10.225553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:10.226044 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:10.226290 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.226307 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.227881 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.227890 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.229809 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.229829 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.231418 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.231427 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.231670 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.232486 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:10.232518 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:10.236246 (+ 3728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:10.241614 (+ 5368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.241623 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.241630 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.242278 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.242830 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.242838 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.242844 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.243041 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.243969 (+ 928us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:10.243977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:10.244202 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 19:38:10.244387 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 19:38:10.244834 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:38:10.246671 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.224818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41302 (local address 127.14.16.124:44121)
0807 19:38:10.224980 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.224987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.225003 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:38:10.225097 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.225107 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.225215 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.225319 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.226455 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.227666 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.230030 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.230992 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.231119 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.236511 (+ 5392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.236520 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.236532 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.236582 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.241379 (+ 4797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.242456 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.242465 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.242470 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.242653 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.243196 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.243203 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.243209 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.243670 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.243885 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.245882 (+ 1997us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.246183 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 19:38:10.252137 20790 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.16.124:44121: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231"
member_type: NON_VOTER
last_known_addr {
host: "127.14.16.124"
port: 44121
}
attrs {
promote: true
}
}
dest_uuid: "128d4df4ad934462bd6508db5f4fe86f"
cas_config_opid_index: 5
I20260807 19:38:10.253225 20790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } }
I20260807 19:38:10.255995 21418 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 LEARNER]: Refusing update from remote peer 128d4df4ad934462bd6508db5f4fe86f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 19:38:10.257544 21503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:10.260309 21367 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 19:38:10.263278 21558 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d00a05c1214f47fc8dfaafec40e4f231 (127.14.16.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:10.266275 21503 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 1 FOLLOWER]: Refusing update from remote peer 128d4df4ad934462bd6508db5f4fe86f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 19:38:10.267143 21558 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.16.124:44121
I20260807 19:38:10.268034 21369 master.cc:537] Master@127.14.16.124:44121 shutting down...
I20260807 19:38:10.275396 21418 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d00a05c1214f47fc8dfaafec40e4f231 (127.14.16.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:10.281164 21447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:10.281455 21447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: This master's current role is: LEARNER
I20260807 19:38:10.282401 21561 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:10.284343 21558 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f: attempting to promote NON_VOTER 00cff960e6a84647bf857c9b54f4c50a to VOTER
I20260807 19:38:10.284233 21367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:10.284566 21367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:10.285583 21558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } }
I20260807 19:38:10.287842 21418 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 LEARNER]: Refusing update from remote peer 128d4df4ad934462bd6508db5f4fe86f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 19:38:10.287734 21562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: NON_VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: true } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:10.288482 21558 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 19:38:10.288597 21562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:10.290795 21561 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 00cff960e6a84647bf857c9b54f4c50a (127.14.16.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:10.294279 21369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:10.294682 21369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:10.294883 21369 tablet_replica.cc:330] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: stopping tablet replica
I20260807 19:38:10.300948 21558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:10.301273 21558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:10.303224 21369 master.cc:552] Master@127.14.16.124:44121 shutdown complete.
I20260807 19:38:10.303332 21369 master_runner.cc:305] Clearing existing system tablet
I20260807 19:38:10.303850 21369 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:10.303988 21369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:10.305342 21558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:10.305583 21558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:10.306907 21418 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 00cff960e6a84647bf857c9b54f4c50a (127.14.16.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:10.312578 21447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:10.312902 21447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:38:10.316427 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.315842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:44121 (local address 127.0.0.1:41308)
0807 19:38:10.316049 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.316186 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:44121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 19:38:10.316917 20772 consensus_peers.cc:487] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f -> Peer d00a05c1214f47fc8dfaafec40e4f231 (127.14.16.124:44121): Couldn't send request to peer d00a05c1214f47fc8dfaafec40e4f231. Status: Network error: Client connection negotiation failed: client connection to 127.14.16.124:44121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:38:10.318018 21369 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.005s sys 0.001s
I20260807 19:38:10.319528 21566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:10.319855 21369 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:10.320139 21369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260807 19:38:10.320245 21369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/wal
uuid: "d00a05c1214f47fc8dfaafec40e4f231"
format_stamp: "Formatted at 2026-08-07 19:38:10 on dist-test-slave-07c2"
server_key: "ab91898ed567538bc86f27fea740f1f7"
server_key_iv: "fec1614199d1baa089c6fd96fc59abaf"
server_key_version: "encryptionkey@0"
I20260807 19:38:10.323812 21369 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:10.328363 21369 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 19:38:10.328528 21369 log.cc:1196] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 19:38:10.328989 21369 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: Deleting consensus metadata
I20260807 19:38:10.329239 21369 master_runner.cc:315] Copying system tablet from 127.14.16.126:41075
I20260807 19:38:10.331617 21369 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: tablet copy: Beginning tablet copy session from remote peer at address 127.14.16.126:41075
I20260807 19:38:10.352269 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.332330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54300 (local address 127.14.16.126:41075)
0807 19:38:10.332563 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.332573 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.339918 (+ 7345us) server_negotiation.cc:407] Connection header received
0807 19:38:10.340123 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.340129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.340213 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.340327 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.341150 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.341865 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.343139 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.343801 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.343919 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.346667 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.346674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.346680 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.346718 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.349634 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.350275 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.350280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.350288 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.350416 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.350806 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.350812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.350815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.351079 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.351255 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.351681 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.351901 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 19:38:10.352340 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.332324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54300)
0807 19:38:10.339797 (+ 7473us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.339818 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:10.339982 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:10.340353 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:10.340358 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:10.340788 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:10.340991 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.341000 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.342029 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.342035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.342991 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.343002 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.344116 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.344123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.344327 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.345371 (+ 1044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:10.345393 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:10.346434 (+ 1041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:10.349777 (+ 3343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.349782 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.349786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.350140 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.350527 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.350532 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.350536 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.350687 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.351201 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:10.351208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:10.351437 (+ 229us) client_negotiation.cc:769] Sending connection context
0807 19:38:10.351583 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 19:38:10.351913 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7341,"thread_start_us":7174,"threads_started":1}
I20260807 19:38:10.353189 20811 tablet_copy_service.cc:140] P 128d4df4ad934462bd6508db5f4fe86f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d00a05c1214f47fc8dfaafec40e4f231 ({username='oryx', principal='oryx/127.14.16.124@KRBTEST.COM'} at 127.0.0.1:54300)
I20260807 19:38:10.353363 20811 tablet_copy_service.cc:161] P 128d4df4ad934462bd6508db5f4fe86f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d00a05c1214f47fc8dfaafec40e4f231 at {username='oryx', principal='oryx/127.14.16.124@KRBTEST.COM'} at 127.0.0.1:54300: session id = d00a05c1214f47fc8dfaafec40e4f231-00000000000000000000000000000000
I20260807 19:38:10.355705 20811 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:38:10.357293 21369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 19:38:10.362630 21369 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: tablet copy: Starting download of 0 data blocks...
I20260807 19:38:10.362972 21369 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: tablet copy: Starting download of 1 WAL segments...
I20260807 19:38:10.365401 21369 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:38:10.368556 20811 tablet_copy_service.cc:342] P 128d4df4ad934462bd6508db5f4fe86f: Request end of tablet copy session d00a05c1214f47fc8dfaafec40e4f231-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.16.124@KRBTEST.COM'} at 127.0.0.1:54300
I20260807 19:38:10.368686 20811 tablet_copy_service.cc:433] P 128d4df4ad934462bd6508db5f4fe86f: ending tablet copy session d00a05c1214f47fc8dfaafec40e4f231-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d00a05c1214f47fc8dfaafec40e4f231
I20260807 19:38:10.379705 21369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:10.382354 21573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:10.384660 21574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:10.403504 21369 server_base.cc:1034] running on GCE node
W20260807 19:38:10.404492 21577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:10.404927 21369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:10.405601 21369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:10.406872 21369 hybrid_clock.cc:648] HybridClock initialized: now 1786131490406826 us; error 31 us; skew 500 ppm
I20260807 19:38:10.407099 21369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131490, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:10.409907 21369 init.cc:377] Logged in from keytab as oryx/127.14.16.124@KRBTEST.COM (short username oryx)
I20260807 19:38:10.412048 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 20855
I20260807 19:38:10.415616 21369 webserver.cc:460] Webserver started at http://127.14.16.124:34789/ using document root <none> and password file <none>
I20260807 19:38:10.415987 21369 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:10.416103 21369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:10.419605 21369 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260807 19:38:10.421712 21584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:10.422194 21369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260807 19:38:10.422327 21369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/wal
uuid: "d00a05c1214f47fc8dfaafec40e4f231"
format_stamp: "Formatted at 2026-08-07 19:38:10 on dist-test-slave-07c2"
server_key: "ab91898ed567538bc86f27fea740f1f7"
server_key_iv: "fec1614199d1baa089c6fd96fc59abaf"
server_key_version: "encryptionkey@0"
I20260807 19:38:10.422525 21369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:10.438583 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:38289
--local_ip_for_outbound_sockets=127.14.16.65
--tserver_master_addrs=127.14.16.126:41075
--webserver_port=33889
--webserver_interface=127.14.16.65
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab
--principal=oryx/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.16.126:41075,127.14.16.125:38459
--tserver_master_addrs=127.14.16.126:41075,127.14.16.125:38459,127.14.16.124:44121 with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:38:10.440631 21369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:10.444166 21369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:10.469327 21369 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.124:44121
I20260807 19:38:10.469367 21637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.124:44121 every 8 connection(s)
I20260807 19:38:10.470589 21369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
I20260807 19:38:10.476248 21639 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:38:10.479736 21639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: Bootstrap starting.
I20260807 19:38:10.498255 21639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:38:10.498772 21639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: Bootstrap complete.
I20260807 19:38:10.499971 21639 raft_consensus.cc:348] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } }
I20260807 19:38:10.500255 21639 raft_consensus.cc:729] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d00a05c1214f47fc8dfaafec40e4f231, State: Initialized, Role: LEARNER
I20260807 19:38:10.500526 21639 consensus_queue.cc:260] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } }
I20260807 19:38:10.501570 21639 sys_catalog.cc:564] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:38:10.501405 21642 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:10.501926 21642 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: This master's current role is: LEARNER
I20260807 19:38:10.518193 21653 catalog_manager.cc:1259] Loaded cluster ID: 542a815e91c4463a8da4ecf60e276a25
I20260807 19:38:10.518325 21653 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: loading cluster ID for follower catalog manager: success
I20260807 19:38:10.521615 21653 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: acquiring CA information for follower catalog manager: success
I20260807 19:38:10.523908 21653 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 19:38:10.801787 21591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:10.802107 21591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:10.802294 21591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:10.809597 21591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:10.809752 21591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:10.809922 21591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:38:10.820109 21591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab
--principal=oryx/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:38289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=33889
--tserver_master_addrs=127.14.16.126:41075,127.14.16.125:38459,127.14.16.124:44121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:10.821381 21591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:10.822724 21591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:10.836498 21660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:10.836535 21662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:10.838511 21659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:10.838637 21591 server_base.cc:1034] running on GCE node
I20260807 19:38:10.839473 21591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:10.840384 21591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:10.843739 21591 hybrid_clock.cc:648] HybridClock initialized: now 1786131490843034 us; error 669 us; skew 500 ppm
I20260807 19:38:10.844110 21591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:38:10.844347 21665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.835593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41324 (local address 127.14.16.124:44121)
0807 19:38:10.838111 (+ 2518us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.838121 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.838463 (+ 342us) server_negotiation.cc:407] Connection header received
0807 19:38:10.838762 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.838770 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.838875 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.838988 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:10.839497 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.840806 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.842514 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.843395 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.843499 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.843639 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.843935 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.844038 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2323,"thread_start_us":2211,"threads_started":1}
I20260807 19:38:10.844393 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.835407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:44121 (local address 127.0.0.1:41324)
0807 19:38:10.838343 (+ 2936us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.838363 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:10.838516 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:10.839044 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:10.839049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:10.839088 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:10.839349 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.839365 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.840960 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.840966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.842377 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.842389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.843558 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.843562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.843727 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.843775 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 19:38:10.843876 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 19:38:10.844013 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2769,"thread_start_us":2660,"threads_started":1}
I20260807 19:38:10.845335 21612 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 19:38:10.845970 21642 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 128d4df4ad934462bd6508db5f4fe86f. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:10.846297 21642 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: This master's current role is: LEARNER
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131490, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:10.847795 21591 init.cc:377] Logged in from keytab as oryx/127.14.16.65@KRBTEST.COM (short username oryx)
I20260807 19:38:10.849210 21591 webserver.cc:460] Webserver started at http://127.14.16.65:33889/ using document root <none> and password file <none>
I20260807 19:38:10.849823 21591 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:10.849936 21591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:10.855067 21591 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:38:10.857846 21671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:10.859447 21591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:38:10.859649 21591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "adcee9bcb55c4940b2cbce43184628df"
format_stamp: "Formatted at 2026-08-07 19:38:07 on dist-test-slave-07c2"
server_key: "d73747f6ee427cd6a2b54a9bdff18bae"
server_key_iv: "b782bb3be23f8170effa4f17255e2476"
server_key_version: "encryptionkey@0"
I20260807 19:38:10.860935 21591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:10.898020 21591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:10.901997 21591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:10.902231 21591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:10.903182 21591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:10.904506 21591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:10.904594 21591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:10.904677 21591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:10.904736 21591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:10.938949 21591 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:38289
I20260807 19:38:10.939041 21787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:38289 every 8 connection(s)
I20260807 19:38:10.940390 21591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:38:10.942570 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 21591
I20260807 19:38:10.942960 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 20995
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131490, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.124@KRBTEST.COM
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131490, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.125@KRBTEST.COM
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131490, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
Aug 07 19:38:10 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.16.66@KRBTEST.COM
I20260807 19:38:10.980039 21665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.944298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:45721 (local address 127.14.16.124:44121)
0807 19:38:10.944518 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.944528 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.958993 (+ 14465us) server_negotiation.cc:407] Connection header received
0807 19:38:10.959758 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.959767 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.959883 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.959972 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.962658 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.963795 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.966379 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.966997 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.967095 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.972299 (+ 5204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.972329 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.972335 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.972388 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.976176 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.977207 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.977212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.977217 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.977381 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.977908 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.977913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.977916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.978303 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.978420 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.979461 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.979690 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:38:10.981308 21798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.958213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:38459 (local address 127.14.16.65:60321)
0807 19:38:10.959451 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.959468 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:10.959652 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:10.960477 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:10.960483 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:10.962636 (+ 2153us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:10.963345 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.963357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.965472 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.965478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.967126 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.967140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.968142 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.968149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.968316 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.969088 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:10.969115 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:10.972058 (+ 2943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:10.976950 (+ 4892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.976956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.976959 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.977475 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.977906 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.977910 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.977914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.978081 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.978793 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:10.978798 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:10.978964 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 19:38:10.979332 (+ 368us) client_negotiation.cc:240] Negotiation successful
0807 19:38:10.979844 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1103,"mutex_wait_us":3,"spinlock_wait_cycles":30848,"thread_start_us":329,"threads_started":1}
I20260807 19:38:10.982020 21796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.944656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.65:43499)
0807 19:38:10.956636 (+ 11980us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.956706 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:10.959381 (+ 2675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:10.961383 (+ 2002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:10.961388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:10.961843 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:10.962391 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.962412 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.963957 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.963967 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.966207 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.966221 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.967506 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.967514 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.967860 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.970669 (+ 2809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:10.970698 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:10.973604 (+ 2906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:10.979060 (+ 5456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.979065 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.979068 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.979567 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.980036 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.980043 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.980049 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.980251 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.980972 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:10.980976 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:10.981161 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 19:38:10.981321 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 19:38:10.981639 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11613,"mutex_wait_us":643,"thread_start_us":11314,"threads_started":1}
I20260807 19:38:10.982803 21799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.957678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:60321 (local address 127.14.16.125:38459)
0807 19:38:10.958238 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.958243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.959552 (+ 1309us) server_negotiation.cc:407] Connection header received
0807 19:38:10.959836 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.959841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.959937 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.960021 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.963519 (+ 3498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.964720 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.967316 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.967974 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.968104 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.973401 (+ 5297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.973408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.973413 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.973456 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.976815 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.977615 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.977620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.977623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.977775 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.978214 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.978219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.978222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.978580 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.978733 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.982030 (+ 3297us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.982407 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":312,"threads_started":1}
I20260807 19:38:10.982904 21794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.943944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:43499 (local address 127.14.16.126:41075)
0807 19:38:10.957604 (+ 13660us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.957611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.957635 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:38:10.959920 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.959925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.960006 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.961527 (+ 1521us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.962584 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.963769 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.966380 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.967033 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.967147 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.973832 (+ 6685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.973840 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.973848 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.973893 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.978877 (+ 4984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.979716 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.979732 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.979735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.979890 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.980381 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.980386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.980389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.980764 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.980906 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.982411 (+ 1505us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.982607 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13535,"thread_start_us":13430,"threads_started":1}
I20260807 19:38:10.981328 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.945313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:44121 (local address 127.14.16.65:45721)
0807 19:38:10.958853 (+ 13540us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:10.958881 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:10.959388 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:10.960473 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:10.960482 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:10.961603 (+ 1121us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:10.962433 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.962457 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.964003 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.964008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.966200 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:10.966214 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.967507 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.967513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:10.967868 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.968836 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:10.968871 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:10.972003 (+ 3132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:10.976305 (+ 4302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.976316 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.976333 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.977072 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.977524 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:10.977531 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:10.977537 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:10.977769 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:10.978529 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:10.978537 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:10.978964 (+ 427us) client_negotiation.cc:769] Sending connection context
0807 19:38:10.980324 (+ 1360us) client_negotiation.cc:240] Negotiation successful
0807 19:38:10.980593 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13379,"mutex_wait_us":177,"spinlock_wait_cycles":3200,"thread_start_us":13554,"threads_started":1}
I20260807 19:38:10.984031 21795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:10.944281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33708 (local address 127.14.16.66:35213)
0807 19:38:10.960506 (+ 16225us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:10.960514 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:10.960539 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:38:10.960604 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:10.960612 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:10.960877 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:10.961054 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:10.963240 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.964849 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.966805 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:10.967678 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:10.967803 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:10.975370 (+ 7567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:10.975389 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:10.975410 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:10.975462 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:10.979564 (+ 4102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.980777 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.980785 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.980789 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.980958 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:10.981439 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:10.981444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:10.981447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:10.981871 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:10.982194 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:10.983435 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0807 19:38:10.983713 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16053,"thread_start_us":15887,"threads_started":1}
I20260807 19:38:10.984398 21791 heartbeater.cc:347] Connected to a master server at 127.14.16.125:38459
I20260807 19:38:10.984607 21789 heartbeater.cc:347] Connected to a master server at 127.14.16.124:44121
I20260807 19:38:10.984763 21791 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:10.984772 21789 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:10.984627 21790 heartbeater.cc:347] Connected to a master server at 127.14.16.126:41075
I20260807 19:38:10.984961 21790 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:10.985456 21121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:10.985909 21790 heartbeater.cc:511] Master 127.14.16.126:41075 requested a full tablet report, sending...
I20260807 19:38:10.985921 21789 heartbeater.cc:511] Master 127.14.16.124:44121 requested a full tablet report, sending...
I20260807 19:38:10.986819 21791 heartbeater.cc:511] Master 127.14.16.125:38459 requested a full tablet report, sending...
I20260807 19:38:10.987809 20790 ts_manager.cc:194] Re-registered known tserver with Master: adcee9bcb55c4940b2cbce43184628df (127.14.16.65:38289)
I20260807 19:38:10.988435 21408 ts_manager.cc:194] Registered new tserver with Master: adcee9bcb55c4940b2cbce43184628df (127.14.16.65:38289)
I20260807 19:38:10.988507 21602 ts_manager.cc:194] Registered new tserver with Master: adcee9bcb55c4940b2cbce43184628df (127.14.16.65:38289)
I20260807 19:38:10.988945 20790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:43499
I20260807 19:38:11.073148 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 20995
I20260807 19:38:11.093065 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:35213
--local_ip_for_outbound_sockets=127.14.16.66
--tserver_master_addrs=127.14.16.126:41075
--webserver_port=34897
--webserver_interface=127.14.16.66
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab
--principal=oryx/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.16.126:41075,127.14.16.125:38459
--tserver_master_addrs=127.14.16.126:41075,127.14.16.125:38459,127.14.16.124:44121 with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:38:11.290594 21367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: NON_VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: true } } }
I20260807 19:38:11.290931 21367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:11.292238 21367 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f: attempting to promote NON_VOTER d00a05c1214f47fc8dfaafec40e4f231 to VOTER
I20260807 19:38:11.292982 21367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } }
I20260807 19:38:11.294903 21612 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 1 LEARNER]: Refusing update from remote peer 128d4df4ad934462bd6508db5f4fe86f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 19:38:11.294909 21418 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 FOLLOWER]: Refusing update from remote peer 128d4df4ad934462bd6508db5f4fe86f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 19:38:11.295585 21367 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [LEADER]: Connected to new peer: Peer: permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 19:38:11.295974 21803 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 19:38:11.299196 21418 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d00a05c1214f47fc8dfaafec40e4f231 (127.14.16.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } } }
I20260807 19:38:11.301069 21612 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d00a05c1214f47fc8dfaafec40e4f231 (127.14.16.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } } }
I20260807 19:38:11.298565 21367 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d00a05c1214f47fc8dfaafec40e4f231 (127.14.16.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } } }
I20260807 19:38:11.304792 21642 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } } }
I20260807 19:38:11.305011 21642 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:38:11.307425 21804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } } }
I20260807 19:38:11.307735 21804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:11.307595 21367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } } }
I20260807 19:38:11.307900 21367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:11.308017 21804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } } }
I20260807 19:38:11.308207 21804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:11.311748 21807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "128d4df4ad934462bd6508db5f4fe86f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } } }
I20260807 19:38:11.312247 21807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: This master's current role is: FOLLOWER
W20260807 19:38:11.441175 21802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:11.441454 21802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:11.441571 21802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:11.449608 21802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:11.449723 21802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:11.449846 21802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:38:11.460736 21802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab
--principal=oryx/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:35213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=34897
--tserver_master_addrs=127.14.16.126:41075,127.14.16.125:38459,127.14.16.124:44121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:11.462092 21802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:11.463521 21802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:11.479130 21813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:11.479346 21815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:11.479969 21802 server_base.cc:1034] running on GCE node
W20260807 19:38:11.482376 21812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:11.483283 21802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:11.484579 21802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:11.485802 21802 hybrid_clock.cc:648] HybridClock initialized: now 1786131491485749 us; error 55 us; skew 500 ppm
I20260807 19:38:11.486166 21802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:11 dist-test-slave-07c2 krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131491, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:11.489822 21802 init.cc:377] Logged in from keytab as oryx/127.14.16.66@KRBTEST.COM (short username oryx)
I20260807 19:38:11.491508 21802 webserver.cc:460] Webserver started at http://127.14.16.66:34897/ using document root <none> and password file <none>
I20260807 19:38:11.492264 21802 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:11.492424 21802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:11.497996 21802 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 19:38:11.500685 21822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:11.502133 21802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 19:38:11.502272 21802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "bf7ee3db77e843c982f7c410345536f8"
format_stamp: "Formatted at 2026-08-07 19:38:08 on dist-test-slave-07c2"
server_key: "cb63eea0b0d0f06c29ca32f34d3c9e8c"
server_key_iv: "a8531d37e626134acec850415887705e"
server_key_version: "encryptionkey@0"
I20260807 19:38:11.503208 21802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:11.518875 21802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:11.522668 21802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:11.523016 21802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:11.524161 21802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:11.525560 21802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:11.525647 21802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:11.525739 21802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:11.525801 21802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:11.559954 21802 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:35213
I20260807 19:38:11.560037 21937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:35213 every 8 connection(s)
I20260807 19:38:11.561426 21802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:38:11.567224 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 21802
I20260807 19:38:11.567616 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 21131
Aug 07 19:38:11 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131491, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.124@KRBTEST.COM
Aug 07 19:38:11 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131491, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
Aug 07 19:38:11 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131491, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.125@KRBTEST.COM
I20260807 19:38:11.597379 21943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:11.564315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:35009 (local address 127.14.16.124:44121)
0807 19:38:11.564886 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:11.564894 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:11.566518 (+ 1624us) server_negotiation.cc:407] Connection header received
0807 19:38:11.570044 (+ 3526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:11.570053 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:11.570168 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:11.570269 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:11.574805 (+ 4536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.576377 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.579522 (+ 3145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.580174 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.580278 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:11.589087 (+ 8809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:11.589094 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:11.589099 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:11.589140 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:11.593358 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:11.594404 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:11.594410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:11.594414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:11.594563 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:11.595120 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:11.595125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:11.595127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:11.595477 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:11.595587 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:11.596710 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0807 19:38:11.596943 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":297,"threads_started":1}
I20260807 19:38:11.599076 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:11.564307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:34881 (local address 127.14.16.126:41075)
0807 19:38:11.565124 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:11.565131 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:11.566210 (+ 1079us) server_negotiation.cc:407] Connection header received
0807 19:38:11.569779 (+ 3569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:11.569788 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:11.569907 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:11.570053 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:11.579802 (+ 9749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.581297 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.583111 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.584006 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.584144 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:11.591201 (+ 7057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:11.591208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:11.591215 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:11.591260 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:11.594973 (+ 3713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:11.595794 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:11.595799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:11.595803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:11.595966 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:11.596464 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:11.596468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:11.596472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:11.596938 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:11.597065 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:11.597932 (+ 867us) server_negotiation.cc:299] Negotiation successful
0807 19:38:11.598718 (+ 786us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":521,"threads_started":1}
Aug 07 19:38:11 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.16.67@KRBTEST.COM
I20260807 19:38:11.599767 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:11.565023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.66:34881)
0807 19:38:11.566036 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:11.566085 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:11.569416 (+ 3331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:11.571053 (+ 1637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:11.571060 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:11.572740 (+ 1680us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:11.573721 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:11.573761 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.581449 (+ 7688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.581455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:11.582956 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:11.582971 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.585221 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.585227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:11.585524 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:11.586885 (+ 1361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:11.586910 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:11.590168 (+ 3258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:11.595102 (+ 4934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:11.595107 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:11.595110 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:11.595614 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:11.596107 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:11.596111 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:11.596114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:11.596316 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:11.597136 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:11.597141 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:11.597298 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 19:38:11.597461 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 19:38:11.598357 (+ 896us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":374,"spinlock_wait_cycles":122368,"thread_start_us":424,"threads_started":1}
I20260807 19:38:11.599803 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:11.565199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:44121 (local address 127.14.16.66:35009)
0807 19:38:11.566078 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:11.566097 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:11.569848 (+ 3751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:11.570854 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:11.570870 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:11.574171 (+ 3301us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:11.574566 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:11.574582 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.576636 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.576644 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:11.579331 (+ 2687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:11.579346 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.580643 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.580653 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:11.580952 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:11.583491 (+ 2539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:11.583523 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:11.586625 (+ 3102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:11.593536 (+ 6911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:11.593549 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:11.593566 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:11.594256 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:11.594720 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:11.594725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:11.594729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:11.594982 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:11.595662 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:11.595689 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:11.596125 (+ 436us) client_negotiation.cc:769] Sending connection context
0807 19:38:11.597618 (+ 1493us) client_negotiation.cc:240] Negotiation successful
0807 19:38:11.598197 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":23,"spinlock_wait_cycles":1280,"thread_start_us":437,"threads_started":1}
I20260807 19:38:11.601398 21939 heartbeater.cc:347] Connected to a master server at 127.14.16.124:44121
I20260807 19:38:11.601722 21939 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:11.601935 21940 heartbeater.cc:347] Connected to a master server at 127.14.16.126:41075
I20260807 19:38:11.602154 21940 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:11.602175 21948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:11.566853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:54627 (local address 127.14.16.125:38459)
0807 19:38:11.568462 (+ 1609us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:11.568468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:11.583159 (+ 14691us) server_negotiation.cc:407] Connection header received
0807 19:38:11.583995 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:11.584003 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:11.584096 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:11.584183 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:11.585717 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.586835 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.588717 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.589207 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.589354 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:11.593979 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:11.593986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:11.593993 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:11.594053 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:11.598209 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:11.599057 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:11.599064 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:11.599069 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:11.599497 (+ 428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:11.600351 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:11.600357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:11.600360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:11.600744 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:11.600885 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:11.601452 (+ 567us) server_negotiation.cc:299] Negotiation successful
0807 19:38:11.601735 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1489,"thread_start_us":1743,"threads_started":1}
I20260807 19:38:11.603098 21940 heartbeater.cc:511] Master 127.14.16.126:41075 requested a full tablet report, sending...
I20260807 19:38:11.603236 21947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:11.566678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:38459 (local address 127.14.16.66:54627)
0807 19:38:11.579574 (+ 12896us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:11.579594 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:11.583205 (+ 3611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:11.584424 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:11.584430 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:11.585055 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:11.585522 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:11.585534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.587019 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.587026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:11.588570 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:11.588582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.589392 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.589399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:11.589613 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:11.590279 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:11.590304 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:11.593761 (+ 3457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:11.598373 (+ 4612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:11.598380 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:11.598384 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:11.598902 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:11.599677 (+ 775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:11.599682 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:11.599686 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:11.600058 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:11.600970 (+ 912us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:11.600974 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:11.601119 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 19:38:11.602453 (+ 1334us) client_negotiation.cc:240] Negotiation successful
0807 19:38:11.602764 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12773,"mutex_wait_us":29,"thread_start_us":12410,"threads_started":1}
I20260807 19:38:11.603493 21939 heartbeater.cc:511] Master 127.14.16.124:44121 requested a full tablet report, sending...
I20260807 19:38:11.604175 21941 heartbeater.cc:347] Connected to a master server at 127.14.16.125:38459
I20260807 19:38:11.604315 21941 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:11.604837 21941 heartbeater.cc:511] Master 127.14.16.125:38459 requested a full tablet report, sending...
I20260807 19:38:11.605535 20790 ts_manager.cc:194] Re-registered known tserver with Master: bf7ee3db77e843c982f7c410345536f8 (127.14.16.66:35213)
I20260807 19:38:11.605662 21602 ts_manager.cc:194] Registered new tserver with Master: bf7ee3db77e843c982f7c410345536f8 (127.14.16.66:35213)
I20260807 19:38:11.605969 21408 ts_manager.cc:194] Registered new tserver with Master: bf7ee3db77e843c982f7c410345536f8 (127.14.16.66:35213)
I20260807 19:38:11.606551 20790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:34881
I20260807 19:38:11.610308 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:11.568890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48532 (local address 127.14.16.67:37913)
0807 19:38:11.584451 (+ 15561us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:11.584464 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:11.585935 (+ 1471us) server_negotiation.cc:407] Connection header received
0807 19:38:11.586263 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:11.586275 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:11.586485 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:11.586651 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:11.588267 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.590344 (+ 2077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.592577 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:11.593488 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:11.593624 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:11.600980 (+ 7356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:11.600995 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:11.601010 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:11.601062 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:11.607516 (+ 6454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:11.608360 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:11.608366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:11.608368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:11.608499 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:11.608930 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:11.608933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:11.608935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:11.609231 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:11.609457 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:11.609722 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 19:38:11.609913 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15339,"thread_start_us":15572,"threads_started":1}
I20260807 19:38:11.611490 21257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:11.695358 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 21131
I20260807 19:38:11.715873 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:37913
--local_ip_for_outbound_sockets=127.14.16.67
--tserver_master_addrs=127.14.16.126:41075
--webserver_port=39627
--webserver_interface=127.14.16.67
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab
--principal=oryx/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.16.126:41075,127.14.16.125:38459
--tserver_master_addrs=127.14.16.126:41075,127.14.16.125:38459,127.14.16.124:44121 with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:38:11.991590 21790 heartbeater.cc:503] Master 127.14.16.126:41075 was elected leader, sending a full tablet report...
W20260807 19:38:12.052680 21952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:12.052953 21952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:12.053048 21952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:12.060312 21952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:12.060417 21952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:12.060535 21952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:38:12.070256 21952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab
--principal=oryx/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:37913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=39627
--tserver_master_addrs=127.14.16.126:41075,127.14.16.125:38459,127.14.16.124:44121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:12.071630 21952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:12.073042 21952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:12.086205 21957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:12.086187 21960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:12.086239 21958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:12.087064 21952 server_base.cc:1034] running on GCE node
I20260807 19:38:12.087669 21952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:12.088553 21952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:12.089797 21952 hybrid_clock.cc:648] HybridClock initialized: now 1786131492089730 us; error 31 us; skew 500 ppm
I20260807 19:38:12.090257 21952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:12 dist-test-slave-07c2 krb5kdc[20742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131492, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:12.093494 21952 init.cc:377] Logged in from keytab as oryx/127.14.16.67@KRBTEST.COM (short username oryx)
I20260807 19:38:12.094822 21952 webserver.cc:460] Webserver started at http://127.14.16.67:39627/ using document root <none> and password file <none>
I20260807 19:38:12.095517 21952 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:12.095635 21952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:12.100898 21952 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 19:38:12.103678 21967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:12.105031 21952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 19:38:12.105198 21952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "b170bb276e7c4f1cbe39f4c8208f7e69"
format_stamp: "Formatted at 2026-08-07 19:38:08 on dist-test-slave-07c2"
server_key: "058ed042b000c34c78db312a61de9248"
server_key_iv: "49cebe84ccca878f52020da0681df78c"
server_key_version: "encryptionkey@0"
I20260807 19:38:12.106148 21952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:12.128830 21952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:12.132637 21952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:12.132931 21952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:12.133909 21952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:12.135233 21952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:12.135389 21952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:12.135525 21952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:12.135622 21952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:12.169600 21952 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:37913
I20260807 19:38:12.169703 22082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:37913 every 8 connection(s)
I20260807 19:38:12.171156 21952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:38:12.179805 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 21952
Aug 07 19:38:12 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131492, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.124@KRBTEST.COM
I20260807 19:38:12.217279 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.174115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:41809 (local address 127.14.16.124:44121)
0807 19:38:12.185047 (+ 10932us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.185057 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.191353 (+ 6296us) server_negotiation.cc:407] Connection header received
0807 19:38:12.191653 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.191663 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.191793 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.191895 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:12.194972 (+ 3077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.196436 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.199483 (+ 3047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.200290 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.200414 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.206347 (+ 5933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:12.206357 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:12.206365 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:12.206422 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:12.211782 (+ 5360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.212977 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.212986 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.212991 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.213159 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.213772 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.213780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.213786 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.214240 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:12.214408 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.216413 (+ 2005us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.216715 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10752,"thread_start_us":10930,"threads_started":1}
I20260807 19:38:12.219897 22091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.174812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:44121 (local address 127.14.16.67:41809)
0807 19:38:12.187942 (+ 13130us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:12.187996 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:12.191149 (+ 3153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:12.192569 (+ 1420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:12.192585 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:12.193576 (+ 991us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:12.194716 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.194750 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.196679 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.196686 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.199232 (+ 2546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.199267 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.200819 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.200830 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.201202 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.202466 (+ 1264us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:12.202502 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:12.206038 (+ 3536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:12.212019 (+ 5981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:12.212030 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:12.212050 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:12.212803 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:12.213308 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:12.213316 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:12.213322 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:12.213610 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:12.214495 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:12.214507 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:12.215034 (+ 527us) client_negotiation.cc:769] Sending connection context
0807 19:38:12.217707 (+ 2673us) client_negotiation.cc:240] Negotiation successful
0807 19:38:12.218285 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12832,"thread_start_us":28708,"threads_started":1}
I20260807 19:38:12.221509 22083 heartbeater.cc:347] Connected to a master server at 127.14.16.124:44121
I20260807 19:38:12.221854 22083 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:12.222836 22083 heartbeater.cc:511] Master 127.14.16.124:44121 requested a full tablet report, sending...
I20260807 19:38:12.224720 21602 ts_manager.cc:194] Registered new tserver with Master: b170bb276e7c4f1cbe39f4c8208f7e69 (127.14.16.67:37913)
Aug 07 19:38:12 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131492, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
Aug 07 19:38:12 dist-test-slave-07c2 krb5kdc[20742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131492, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.125@KRBTEST.COM
I20260807 19:38:12.240088 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.194709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.67:44521)
0807 19:38:12.217925 (+ 23216us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:12.217947 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:12.218069 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:12.218616 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:12.218621 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:12.219211 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:12.219518 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.219531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.222203 (+ 2672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.222208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.224064 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.224078 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.225321 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.225328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.225495 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.226488 (+ 993us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:12.226509 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:12.229298 (+ 2789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:12.234000 (+ 4702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:12.234004 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:12.234007 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:12.234576 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:12.238168 (+ 3592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:12.238172 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:12.238176 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:12.238351 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:12.239081 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:12.239086 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:12.239254 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 19:38:12.239407 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 19:38:12.239695 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9695,"thread_start_us":10414,"threads_started":1}
I20260807 19:38:12.240378 22087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.174067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:43781 (local address 127.14.16.125:38459)
0807 19:38:12.204558 (+ 30491us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.204565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.204586 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:38:12.204650 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.204656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.204734 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.204860 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:12.221019 (+ 16159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.222153 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.227870 (+ 5717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.228598 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.228708 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.232930 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:12.232936 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:12.232941 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:12.232989 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:12.237248 (+ 4259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.238092 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.238099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.238103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.238274 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.238716 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.238732 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.238735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.239089 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:12.239222 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.239746 (+ 524us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.239965 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30358,"thread_start_us":30908,"threads_started":1}
I20260807 19:38:12.241020 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.188712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:44521 (local address 127.14.16.126:41075)
0807 19:38:12.217799 (+ 29087us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.217805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.218175 (+ 370us) server_negotiation.cc:407] Connection header received
0807 19:38:12.218256 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.218261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.218340 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.218439 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:12.219694 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.222032 (+ 2338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.224258 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.224919 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.225045 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.229540 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:12.229548 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:12.229554 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:12.229603 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:12.233823 (+ 4220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.234731 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.234736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.234740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.234876 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.238481 (+ 3605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.238488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.238494 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.238872 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:12.239017 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.240448 (+ 1431us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.240684 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28970,"thread_start_us":34960,"threads_started":1}
I20260807 19:38:12.241384 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.174812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:38459 (local address 127.14.16.67:43781)
0807 19:38:12.204026 (+ 29214us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:12.204057 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:12.204201 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:12.204915 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:12.204920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:12.215876 (+ 10956us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:12.220854 (+ 4978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.220869 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.225717 (+ 4848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.225734 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.227644 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.227667 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.229051 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.229058 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.229243 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.229842 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:12.229866 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:12.232682 (+ 2816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:12.237441 (+ 4759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:12.237445 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:12.237448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:12.237940 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:12.238420 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:12.238426 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:12.238433 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:12.238582 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:12.239317 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:12.239322 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:12.239480 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 19:38:12.240690 (+ 1210us) client_negotiation.cc:240] Negotiation successful
0807 19:38:12.240962 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29022,"mutex_wait_us":38,"spinlock_wait_cycles":47488,"thread_start_us":12522,"threads_started":1}
I20260807 19:38:12.241829 22084 heartbeater.cc:347] Connected to a master server at 127.14.16.126:41075
I20260807 19:38:12.241973 22084 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:12.242416 22084 heartbeater.cc:511] Master 127.14.16.126:41075 requested a full tablet report, sending...
I20260807 19:38:12.243393 22085 heartbeater.cc:347] Connected to a master server at 127.14.16.125:38459
I20260807 19:38:12.243527 22085 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:12.243933 22085 heartbeater.cc:511] Master 127.14.16.125:38459 requested a full tablet report, sending...
I20260807 19:38:12.244621 20790 ts_manager.cc:194] Re-registered known tserver with Master: b170bb276e7c4f1cbe39f4c8208f7e69 (127.14.16.67:37913)
I20260807 19:38:12.245201 21408 ts_manager.cc:194] Registered new tserver with Master: b170bb276e7c4f1cbe39f4c8208f7e69 (127.14.16.67:37913)
I20260807 19:38:12.245499 20790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:44521
I20260807 19:38:12.263322 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.206000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54310 (local address 127.14.16.126:41075)
0807 19:38:12.221842 (+ 15842us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.221850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.226333 (+ 4483us) server_negotiation.cc:407] Connection header received
0807 19:38:12.235277 (+ 8944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.235315 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.235406 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.235499 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:12.242028 (+ 6529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.243018 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.247851 (+ 4833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.248384 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.248457 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.252292 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:12.252300 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:12.252305 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:12.252340 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:12.260228 (+ 7888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.261120 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.261125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.261128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.261298 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.261724 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.261730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.261734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.262004 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:12.262286 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.262642 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.262879 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15708,"thread_start_us":11784,"threads_started":1}
I20260807 19:38:12.267764 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.206106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41332 (local address 127.14.16.124:44121)
0807 19:38:12.217634 (+ 11528us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.217640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.250082 (+ 32442us) server_negotiation.cc:407] Connection header received
0807 19:38:12.250288 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.250295 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.250401 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.250507 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:12.252888 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.254093 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.256121 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.256736 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.256855 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.260638 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:12.260645 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:12.260651 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:12.260692 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:12.264099 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.265132 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.265140 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.265144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.265311 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.266192 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.266198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.266202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.266590 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:12.266852 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.267108 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.267402 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11419,"thread_start_us":49196,"threads_started":1}
I20260807 19:38:12.273653 22087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.226428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56874 (local address 127.14.16.125:38459)
0807 19:38:12.240643 (+ 14215us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.240649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.256020 (+ 15371us) server_negotiation.cc:407] Connection header received
0807 19:38:12.256876 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.256882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.256966 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.257032 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:12.259326 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.260458 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.262308 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.262999 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.263096 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.266632 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:12.266641 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:12.266647 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:12.266687 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:12.270855 (+ 4168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.271611 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.271617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.271620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.271770 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.272181 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.272185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.272187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.272444 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:12.272768 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.273059 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.273299 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14091,"thread_start_us":16568,"threads_started":1}
I20260807 19:38:12.269181 20790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:38:12.278128 20790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:38:12.305819 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.297396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33716 (local address 127.14.16.66:35213)
0807 19:38:12.298502 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.298511 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.298545 (+ 34us) server_negotiation.cc:407] Connection header received
0807 19:38:12.298622 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.298627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.298817 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.299017 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:12.299717 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.301394 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.303524 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.304802 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.304957 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.305159 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.305387 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.305550 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":960,"thread_start_us":853,"threads_started":1}
I20260807 19:38:12.305833 22112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.297138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:35213 (local address 127.0.0.1:33716)
0807 19:38:12.297825 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:12.297850 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:12.298015 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:12.299001 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:12.299006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:12.299032 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:12.299513 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.299533 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.301533 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.301539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.303272 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.303323 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.305007 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.305014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.305183 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.305218 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:38:12.305339 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 19:38:12.305472 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":507,"threads_started":1}
I20260807 19:38:12.310249 21872 tablet_service.cc:1467] Processing CreateTablet for tablet d9aeb4efb2b943888cee30ec9962491d (DEFAULT_TABLE table=test-table [id=1f1d9d40bb2541bd926ec9c4bd39af69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:12.311962 21872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9aeb4efb2b943888cee30ec9962491d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:12.319149 22112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.297590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:38289 (local address 127.0.0.1:54774)
0807 19:38:12.306082 (+ 8492us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:12.306099 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:12.306185 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:12.309801 (+ 3616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:12.309809 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:12.309848 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:12.310088 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.310102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.312067 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.312075 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.314650 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.314672 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.318019 (+ 3347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.318029 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.318303 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.318360 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 19:38:12.318477 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 19:38:12.318681 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8380,"thread_start_us":9870,"threads_started":1}
I20260807 19:38:12.320142 22115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.298098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54774 (local address 127.14.16.65:38289)
0807 19:38:12.309192 (+ 11094us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.309201 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.309231 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:38:12.309325 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.309331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.309564 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.309755 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:12.310332 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.311911 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.316336 (+ 4425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.317807 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.317947 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.318164 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.319477 (+ 1313us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.319713 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10931,"thread_start_us":15971,"threads_started":1}
I20260807 19:38:12.324146 21722 tablet_service.cc:1467] Processing CreateTablet for tablet d9aeb4efb2b943888cee30ec9962491d (DEFAULT_TABLE table=test-table [id=1f1d9d40bb2541bd926ec9c4bd39af69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:12.325569 21722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9aeb4efb2b943888cee30ec9962491d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:12.327204 22114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.308094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:37913 (local address 127.0.0.1:48544)
0807 19:38:12.308594 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:12.308611 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:12.308719 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:12.321430 (+ 12711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:12.321435 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:12.321461 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:12.321784 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.321797 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.323391 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.323399 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.325026 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.325039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.326684 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.326690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.326842 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.326866 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:38:12.326921 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 19:38:12.327007 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358}
I20260807 19:38:12.328106 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.308194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48544 (local address 127.14.16.67:37913)
0807 19:38:12.320874 (+ 12680us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.320882 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.320913 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:38:12.321003 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.321010 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.321245 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.321440 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:12.321988 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.323230 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.325180 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.326480 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.327577 (+ 1097us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.327732 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.327782 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.327882 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12471,"thread_start_us":13441,"threads_started":1}
I20260807 19:38:12.330837 22017 tablet_service.cc:1467] Processing CreateTablet for tablet d9aeb4efb2b943888cee30ec9962491d (DEFAULT_TABLE table=test-table [id=1f1d9d40bb2541bd926ec9c4bd39af69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:12.332113 22117 tablet_bootstrap.cc:492] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8: Bootstrap starting.
I20260807 19:38:12.332162 22017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9aeb4efb2b943888cee30ec9962491d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:12.335815 22117 tablet_bootstrap.cc:654] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:12.337080 22117 log.cc:824] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:12.340242 22117 tablet_bootstrap.cc:492] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8: No bootstrap required, opened a new log
I20260807 19:38:12.340502 22117 ts_tablet_manager.cc:1397] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:38:12.341027 22119 tablet_bootstrap.cc:492] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df: Bootstrap starting.
I20260807 19:38:12.343803 22119 tablet_bootstrap.cc:654] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:12.344822 22119 log.cc:824] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:12.348344 22121 tablet_bootstrap.cc:492] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69: Bootstrap starting.
I20260807 19:38:12.348662 22119 tablet_bootstrap.cc:492] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df: No bootstrap required, opened a new log
I20260807 19:38:12.348941 22119 ts_tablet_manager.cc:1397] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 19:38:12.350442 22117 raft_consensus.cc:348] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } } peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } }
I20260807 19:38:12.350821 22117 raft_consensus.cc:374] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:12.350943 22117 raft_consensus.cc:729] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7ee3db77e843c982f7c410345536f8, State: Initialized, Role: FOLLOWER
I20260807 19:38:12.351538 22117 consensus_queue.cc:260] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } } peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } }
I20260807 19:38:12.351907 22121 tablet_bootstrap.cc:654] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:12.353068 22121 log.cc:824] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:12.353253 22117 ts_tablet_manager.cc:1428] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260807 19:38:12.353502 21940 heartbeater.cc:503] Master 127.14.16.126:41075 was elected leader, sending a full tablet report...
I20260807 19:38:12.361665 22119 raft_consensus.cc:348] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } } peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } }
I20260807 19:38:12.362114 22119 raft_consensus.cc:374] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:12.362234 22119 raft_consensus.cc:729] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adcee9bcb55c4940b2cbce43184628df, State: Initialized, Role: FOLLOWER
I20260807 19:38:12.363004 22121 tablet_bootstrap.cc:492] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69: No bootstrap required, opened a new log
I20260807 19:38:12.362861 22119 consensus_queue.cc:260] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } } peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } }
I20260807 19:38:12.363318 22121 ts_tablet_manager.cc:1397] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260807 19:38:12.364688 22119 ts_tablet_manager.cc:1428] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df: Time spent starting tablet: real 0.016s user 0.008s sys 0.007s
I20260807 19:38:12.373473 22121 raft_consensus.cc:348] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } } peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } }
I20260807 19:38:12.373822 22121 raft_consensus.cc:374] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:12.373922 22121 raft_consensus.cc:729] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b170bb276e7c4f1cbe39f4c8208f7e69, State: Initialized, Role: FOLLOWER
I20260807 19:38:12.374426 22121 consensus_queue.cc:260] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } } peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } }
I20260807 19:38:12.374753 22124 raft_consensus.cc:482] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:38:12.375085 22124 raft_consensus.cc:504] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } } peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } }
I20260807 19:38:12.376343 22121 ts_tablet_manager.cc:1428] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69: Time spent starting tablet: real 0.013s user 0.008s sys 0.002s
I20260807 19:38:12.376654 22084 heartbeater.cc:503] Master 127.14.16.126:41075 was elected leader, sending a full tablet report...
I20260807 19:38:12.380630 22124 leader_election.cc:288] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf7ee3db77e843c982f7c410345536f8 (127.14.16.66:35213), b170bb276e7c4f1cbe39f4c8208f7e69 (127.14.16.67:37913)
I20260807 19:38:12.387005 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.377837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:58195 (local address 127.14.16.66:35213)
0807 19:38:12.378027 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.378033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.379978 (+ 1945us) server_negotiation.cc:407] Connection header received
0807 19:38:12.380301 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.380310 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.380394 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.380483 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:12.381403 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.382481 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.385023 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.386164 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.386336 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.386412 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.386665 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.386799 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:38:12.387199 22126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.377372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:35213 (local address 127.14.16.65:58195)
0807 19:38:12.379873 (+ 2501us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:12.379893 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:12.380102 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:12.380652 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:12.380655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:12.380675 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:12.381220 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.381249 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.382638 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.382641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.384872 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.384887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.386363 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.386367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.386486 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.386509 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:38:12.386642 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 19:38:12.386788 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2369,"mutex_wait_us":24,"thread_start_us":1314,"threads_started":1}
I20260807 19:38:12.387526 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.379684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:45573 (local address 127.14.16.67:37913)
0807 19:38:12.379868 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.379874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.381528 (+ 1654us) server_negotiation.cc:407] Connection header received
0807 19:38:12.381632 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.381638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.381707 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.381808 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:12.382231 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.383075 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.385332 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.386534 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.386700 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.386774 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.387129 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.387319 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 19:38:12.387656 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.379239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:37913 (local address 127.14.16.65:45573)
0807 19:38:12.381420 (+ 2181us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:12.381442 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:12.381543 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:12.381824 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:12.381827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:12.381844 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:12.382079 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.382085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.383194 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.383199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.385195 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:12.385204 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.386758 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.386763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:12.386952 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.386986 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:38:12.387084 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 19:38:12.387228 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2041,"mutex_wait_us":1,"thread_start_us":1794,"threads_started":1}
I20260807 19:38:12.388228 21893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9aeb4efb2b943888cee30ec9962491d" candidate_uuid: "adcee9bcb55c4940b2cbce43184628df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7ee3db77e843c982f7c410345536f8" is_pre_election: true
I20260807 19:38:12.388654 22038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9aeb4efb2b943888cee30ec9962491d" candidate_uuid: "adcee9bcb55c4940b2cbce43184628df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" is_pre_election: true
I20260807 19:38:12.388873 21893 raft_consensus.cc:2454] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adcee9bcb55c4940b2cbce43184628df in term 0.
I20260807 19:38:12.389181 22038 raft_consensus.cc:2454] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adcee9bcb55c4940b2cbce43184628df in term 0.
I20260807 19:38:12.389549 21673 leader_election.cc:302] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adcee9bcb55c4940b2cbce43184628df, bf7ee3db77e843c982f7c410345536f8; no voters:
I20260807 19:38:12.389969 22124 raft_consensus.cc:2781] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:38:12.390115 22124 raft_consensus.cc:482] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:38:12.390235 22124 raft_consensus.cc:3037] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:12.392525 22124 raft_consensus.cc:504] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } } peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } }
I20260807 19:38:12.393100 22124 leader_election.cc:288] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [CANDIDATE]: Term 1 election: Requested vote from peers bf7ee3db77e843c982f7c410345536f8 (127.14.16.66:35213), b170bb276e7c4f1cbe39f4c8208f7e69 (127.14.16.67:37913)
I20260807 19:38:12.393504 22038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9aeb4efb2b943888cee30ec9962491d" candidate_uuid: "adcee9bcb55c4940b2cbce43184628df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69"
I20260807 19:38:12.393519 21893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9aeb4efb2b943888cee30ec9962491d" candidate_uuid: "adcee9bcb55c4940b2cbce43184628df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7ee3db77e843c982f7c410345536f8"
I20260807 19:38:12.393697 22038 raft_consensus.cc:3037] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:12.393716 21893 raft_consensus.cc:3037] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:12.396387 22038 raft_consensus.cc:2454] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adcee9bcb55c4940b2cbce43184628df in term 1.
I20260807 19:38:12.396538 21893 raft_consensus.cc:2454] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adcee9bcb55c4940b2cbce43184628df in term 1.
I20260807 19:38:12.396799 21675 leader_election.cc:302] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adcee9bcb55c4940b2cbce43184628df, b170bb276e7c4f1cbe39f4c8208f7e69; no voters:
I20260807 19:38:12.397171 22124 raft_consensus.cc:2781] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:12.397639 22124 raft_consensus.cc:686] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 1 LEADER]: Becoming Leader. State: Replica: adcee9bcb55c4940b2cbce43184628df, State: Running, Role: LEADER
I20260807 19:38:12.398093 22124 consensus_queue.cc:237] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } } peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } }
I20260807 19:38:12.402976 20790 catalog_manager.cc:5495] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df reported cstate change: term changed from 0 to 1, leader changed from <none> to adcee9bcb55c4940b2cbce43184628df (127.14.16.65). New cstate: current_term: 1 leader_uuid: "adcee9bcb55c4940b2cbce43184628df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } health_report { overall_health: UNKNOWN } } }
W20260807 19:38:12.424136 22086 tablet.cc:2369] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:38:12.431470 22115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.426193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54778 (local address 127.14.16.65:38289)
0807 19:38:12.426410 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.426416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.426435 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:38:12.426568 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.426573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.426650 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.426806 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:38:12.427185 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.427965 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.429279 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.429689 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.429826 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.430315 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:38:12.430802 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:38:12.430961 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.431058 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.431141 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:38:12.439203 22038 raft_consensus.cc:1261] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69 [term 1 FOLLOWER]: Refusing update from remote peer adcee9bcb55c4940b2cbce43184628df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:12.439215 21893 raft_consensus.cc:1261] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8 [term 1 FOLLOWER]: Refusing update from remote peer adcee9bcb55c4940b2cbce43184628df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:12.440202 22124 consensus_queue.cc:1035] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:12.440511 22128 consensus_queue.cc:1035] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:38:12.444494 21792 tablet.cc:2369] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:38:12.454190 22131 mvcc.cc:205] Tried to move back new op lower bound from 7315994593022095360 to 7315994592865935360. Current Snapshot: MvccSnapshot[applied={T|T < 7315994593022095360}]
I20260807 19:38:12.455775 22132 mvcc.cc:205] Tried to move back new op lower bound from 7315994593022095360 to 7315994592865935360. Current Snapshot: MvccSnapshot[applied={T|T < 7315994593022095360}]
I20260807 19:38:12.479058 20790 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:12.479395 20790 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:12.486357 20790 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 128d4df4ad934462bd6508db5f4fe86f: Sending DeleteTablet for 3 replicas of tablet d9aeb4efb2b943888cee30ec9962491d
I20260807 19:38:12.487562 21872 tablet_service.cc:1514] Processing DeleteTablet for tablet d9aeb4efb2b943888cee30ec9962491d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:12 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:33716
I20260807 19:38:12.488526 22141 tablet_replica.cc:330] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8: stopping tablet replica
I20260807 19:38:12.489041 22141 raft_consensus.cc:2229] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:12.489670 21722 tablet_service.cc:1514] Processing DeleteTablet for tablet d9aeb4efb2b943888cee30ec9962491d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:12 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:54774
I20260807 19:38:12.489596 22141 raft_consensus.cc:2258] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:12.490412 22017 tablet_service.cc:1514] Processing DeleteTablet for tablet d9aeb4efb2b943888cee30ec9962491d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:12 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:48544
I20260807 19:38:12.492429 22141 ts_tablet_manager.cc:1905] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:12.492698 22144 tablet_replica.cc:330] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69: stopping tablet replica
I20260807 19:38:12.493160 22144 raft_consensus.cc:2229] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:12.493718 22144 raft_consensus.cc:2258] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:12.495685 22144 ts_tablet_manager.cc:1905] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:12.500368 22141 ts_tablet_manager.cc:1918] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:12.500546 22141 log.cc:1196] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/d9aeb4efb2b943888cee30ec9962491d
I20260807 19:38:12.501297 22141 ts_tablet_manager.cc:1939] T d9aeb4efb2b943888cee30ec9962491d P bf7ee3db77e843c982f7c410345536f8: Deleting consensus metadata
I20260807 19:38:12.503235 20773 catalog_manager.cc:4841] TS bf7ee3db77e843c982f7c410345536f8 (127.14.16.66:35213): tablet d9aeb4efb2b943888cee30ec9962491d (table test-table [id=1f1d9d40bb2541bd926ec9c4bd39af69]) successfully deleted
I20260807 19:38:12.503453 22144 ts_tablet_manager.cc:1918] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:12.503768 22144 log.cc:1196] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/d9aeb4efb2b943888cee30ec9962491d
I20260807 19:38:12.504253 22144 ts_tablet_manager.cc:1939] T d9aeb4efb2b943888cee30ec9962491d P b170bb276e7c4f1cbe39f4c8208f7e69: Deleting consensus metadata
I20260807 19:38:12.505673 20775 catalog_manager.cc:4841] TS b170bb276e7c4f1cbe39f4c8208f7e69 (127.14.16.67:37913): tablet d9aeb4efb2b943888cee30ec9962491d (table test-table [id=1f1d9d40bb2541bd926ec9c4bd39af69]) successfully deleted
I20260807 19:38:12.508836 22143 tablet_replica.cc:330] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df: stopping tablet replica
I20260807 19:38:12.509332 22143 raft_consensus.cc:2229] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:38:12.509455 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 20755
I20260807 19:38:12.510075 22143 raft_consensus.cc:2258] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:12.510666 20822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:12.515477 22143 ts_tablet_manager.cc:1905] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:12.523460 22143 ts_tablet_manager.cc:1918] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:12.523665 22143 log.cc:1196] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/d9aeb4efb2b943888cee30ec9962491d
I20260807 19:38:12.524302 22143 ts_tablet_manager.cc:1939] T d9aeb4efb2b943888cee30ec9962491d P adcee9bcb55c4940b2cbce43184628df: Deleting consensus metadata
I20260807 19:38:12.528604 22087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.491522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56884 (local address 127.14.16.125:38459)
0807 19:38:12.491755 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.491763 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.508220 (+ 16457us) server_negotiation.cc:407] Connection header received
0807 19:38:12.508307 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.508315 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.508429 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.508548 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:12.513343 (+ 4795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.514569 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.516594 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.517234 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.517337 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.521139 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:12.521152 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:12.521157 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:12.521201 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:12.525044 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.526462 (+ 1418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.526468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.526471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.526618 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.527018 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.527023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.527027 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.527389 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:12.527657 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.527971 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.528190 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:38:12.542174 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.508823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.14.16.124:44121)
0807 19:38:12.509038 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.509045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.529349 (+ 20304us) server_negotiation.cc:407] Connection header received
0807 19:38:12.529514 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.529520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.529624 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.529731 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:12.531117 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.532281 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.533459 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.533896 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.534031 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.536629 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:12.536634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:12.536637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:12.536669 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:12.539368 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.540061 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.540068 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.540072 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.540211 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.540599 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.540604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.540607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.540969 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:12.541227 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.541545 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.541779 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:38:12.704115 20775 catalog_manager.cc:4841] TS adcee9bcb55c4940b2cbce43184628df (127.14.16.65:38289): tablet d9aeb4efb2b943888cee30ec9962491d (table test-table [id=1f1d9d40bb2541bd926ec9c4bd39af69]) successfully deleted
I20260807 19:38:12.705299 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 20755
I20260807 19:38:12.760653 22087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.742645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56898 (local address 127.14.16.125:38459)
0807 19:38:12.742842 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.742847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.743783 (+ 936us) server_negotiation.cc:407] Connection header received
0807 19:38:12.744032 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.744039 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.744132 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.744202 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:12.746615 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.747804 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.750048 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.750681 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.750769 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.754622 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:12.754629 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:12.754636 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:12.754684 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:12.757808 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.758574 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.758580 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.758584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.758719 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.759100 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.759105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.759109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.759478 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:12.759759 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.760070 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.760301 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 19:38:12.762861 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:12.741530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41350 (local address 127.14.16.124:44121)
0807 19:38:12.741804 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:12.741810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:12.743946 (+ 2136us) server_negotiation.cc:407] Connection header received
0807 19:38:12.744039 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:12.744047 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:12.744153 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:12.744260 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:12.747311 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.748812 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.751478 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:12.752282 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:12.752411 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:12.756158 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:12.756165 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:12.756170 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:12.756209 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:12.759635 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.760519 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.760524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.760527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.760669 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:12.761182 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:12.761187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:12.761190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:12.761591 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:12.761839 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:12.762316 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 19:38:12.762541 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:38:13.463109 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:13.462155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.66:42929)
0807 19:38:13.462669 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:13.462834 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":260,"threads_started":1}
W20260807 19:38:13.463636 21940 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:41075 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
I20260807 19:38:13.468621 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:13.467870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.67:38441)
0807 19:38:13.468304 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:13.468420 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":215,"threads_started":1}
W20260807 19:38:13.469061 22084 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:41075 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
I20260807 19:38:13.474579 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:13.473760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.65:38851)
0807 19:38:13.474220 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:13.474369 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":235,"threads_started":1}
W20260807 19:38:13.474951 21790 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:41075 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
I20260807 19:38:13.987371 22157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 128d4df4ad934462bd6508db5f4fe86f)
I20260807 19:38:13.987380 22158 raft_consensus.cc:482] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 128d4df4ad934462bd6508db5f4fe86f)
I20260807 19:38:13.987659 22158 raft_consensus.cc:504] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } }
I20260807 19:38:13.987699 22157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } }
I20260807 19:38:13.989151 22158 leader_election.cc:288] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 128d4df4ad934462bd6508db5f4fe86f (127.14.16.126:41075), 00cff960e6a84647bf857c9b54f4c50a (127.14.16.125:38459)
I20260807 19:38:13.991099 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:13.989141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54740)
0807 19:38:13.990494 (+ 1353us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:13.990753 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1175,"thread_start_us":972,"threads_started":1}
W20260807 19:38:13.991568 21395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
I20260807 19:38:13.996434 22157 leader_election.cc:288] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 128d4df4ad934462bd6508db5f4fe86f (127.14.16.126:41075), d00a05c1214f47fc8dfaafec40e4f231 (127.14.16.124:44121)
I20260807 19:38:14.002717 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:13.989630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54746)
0807 19:38:14.001995 (+ 12365us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.002318 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12173,"thread_start_us":11976,"threads_started":1}
I20260807 19:38:14.003429 22161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.001282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54756)
0807 19:38:14.002930 (+ 1648us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.003144 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1195}
W20260807 19:38:14.003841 21588 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
W20260807 19:38:14.004240 21395 leader_election.cc:334] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 128d4df4ad934462bd6508db5f4fe86f (127.14.16.126:41075): Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
I20260807 19:38:14.006467 22164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.005787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54758)
0807 19:38:14.006015 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.006225 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260807 19:38:14.006767 21588 leader_election.cc:334] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 128d4df4ad934462bd6508db5f4fe86f (127.14.16.126:41075): Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
I20260807 19:38:14.008183 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:13.990707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:44121 (local address 127.0.0.1:41364)
0807 19:38:13.996356 (+ 5649us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:13.996380 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:13.996544 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:14.000988 (+ 4444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:14.000996 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:14.001035 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:14.001367 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.001384 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.003315 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.003322 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.005470 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.005518 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.007421 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.007426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.007566 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.007592 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:38:14.007759 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 19:38:14.007907 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5501,"thread_start_us":11642,"threads_started":1}
I20260807 19:38:14.008378 22162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:13.990884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41364 (local address 127.14.16.124:44121)
0807 19:38:14.000369 (+ 9485us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:14.000375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:14.000395 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:38:14.000472 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:14.000478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:14.000577 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:14.000699 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:14.001608 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.003109 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.005707 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.007255 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.007393 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.007566 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:14.007841 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 19:38:14.008007 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":9328,"thread_start_us":9186,"threads_started":1}
I20260807 19:38:14.009730 21612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00cff960e6a84647bf857c9b54f4c50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d00a05c1214f47fc8dfaafec40e4f231" is_pre_election: true
I20260807 19:38:14.010089 21612 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00cff960e6a84647bf857c9b54f4c50a in term 1.
I20260807 19:38:14.010629 21392 leader_election.cc:302] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00cff960e6a84647bf857c9b54f4c50a, d00a05c1214f47fc8dfaafec40e4f231; no voters: 128d4df4ad934462bd6508db5f4fe86f
I20260807 19:38:14.011041 22157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:38:14.011166 22157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 128d4df4ad934462bd6508db5f4fe86f)
I20260807 19:38:14.011281 22157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:38:14.011600 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.002110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:38459 (local address 127.0.0.1:56912)
0807 19:38:14.003000 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.003023 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:14.003173 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:14.004544 (+ 1371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:14.004548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:14.004577 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:14.004946 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.004966 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.006827 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.006834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.008939 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.008955 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.010718 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.010722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.010915 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.010940 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:38:14.011045 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 19:38:14.011244 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":7,"thread_start_us":1447,"threads_started":1}
I20260807 19:38:14.011832 22166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.002634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56912 (local address 127.14.16.125:38459)
0807 19:38:14.004078 (+ 1444us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:14.004084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:14.004109 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:38:14.004167 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:14.004280 (+ 113us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:14.004370 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:14.005115 (+ 745us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:14.005448 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.006658 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.009122 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.010484 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.010686 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.010829 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:14.011073 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 19:38:14.011418 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1290,"thread_start_us":1081,"threads_started":1}
I20260807 19:38:14.012421 21418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d00a05c1214f47fc8dfaafec40e4f231" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "00cff960e6a84647bf857c9b54f4c50a" is_pre_election: true
I20260807 19:38:14.014025 22157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } }
I20260807 19:38:14.014453 21418 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d00a05c1214f47fc8dfaafec40e4f231 in current term 2: Already voted for candidate 00cff960e6a84647bf857c9b54f4c50a in this term.
I20260807 19:38:14.014843 22157 leader_election.cc:288] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [CANDIDATE]: Term 2 election: Requested vote from peers 128d4df4ad934462bd6508db5f4fe86f (127.14.16.126:41075), d00a05c1214f47fc8dfaafec40e4f231 (127.14.16.124:44121)
I20260807 19:38:14.015403 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.014916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54766)
0807 19:38:14.015132 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.015222 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260807 19:38:14.015367 21612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00cff960e6a84647bf857c9b54f4c50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d00a05c1214f47fc8dfaafec40e4f231"
I20260807 19:38:14.015575 21612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:38:14.015537 21588 leader_election.cc:302] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d00a05c1214f47fc8dfaafec40e4f231; no voters: 00cff960e6a84647bf857c9b54f4c50a, 128d4df4ad934462bd6508db5f4fe86f
I20260807 19:38:14.016283 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.015960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54770)
0807 19:38:14.016126 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.016192 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260807 19:38:14.016444 21395 leader_election.cc:334] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 128d4df4ad934462bd6508db5f4fe86f (127.14.16.126:41075): Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
I20260807 19:38:14.017817 21612 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cff960e6a84647bf857c9b54f4c50a in term 2.
I20260807 19:38:14.017987 22158 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:38:14.018294 21392 leader_election.cc:302] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 00cff960e6a84647bf857c9b54f4c50a, d00a05c1214f47fc8dfaafec40e4f231; no voters: 128d4df4ad934462bd6508db5f4fe86f
I20260807 19:38:14.018590 22157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:38:14.019006 22157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [term 2 LEADER]: Becoming Leader. State: Replica: 00cff960e6a84647bf857c9b54f4c50a, State: Running, Role: LEADER
I20260807 19:38:14.019439 22157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } }
I20260807 19:38:14.021122 22168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00cff960e6a84647bf857c9b54f4c50a. Latest consensus state: current_term: 2 leader_uuid: "00cff960e6a84647bf857c9b54f4c50a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } } }
I20260807 19:38:14.021456 22168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:14.022083 22170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:38:14.025650 22170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:38:14.026461 22170 catalog_manager.cc:1259] Loaded cluster ID: 542a815e91c4463a8da4ecf60e276a25
I20260807 19:38:14.026562 22170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:38:14.027768 22170 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:38:14.028626 22170 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: Loaded TSK: 0
I20260807 19:38:14.029426 22170 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:38:14.050467 21408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:38:14.057410 21408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:38:14.061416 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.060935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54800)
0807 19:38:14.061134 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.061248 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 19:38:14.061514 21612 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [term 2 FOLLOWER]: Refusing update from remote peer 00cff960e6a84647bf857c9b54f4c50a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 19:38:14.062242 22168 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 19:38:14.062399 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.062046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54802)
0807 19:38:14.062201 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.062275 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 19:38:14.062645 21395 consensus_peers.cc:487] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a -> Peer 128d4df4ad934462bd6508db5f4fe86f (127.14.16.126:41075): Couldn't send request to peer 128d4df4ad934462bd6508db5f4fe86f. Status: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:38:14.064972 22158 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00cff960e6a84647bf857c9b54f4c50a. Latest consensus state: current_term: 2 leader_uuid: "00cff960e6a84647bf857c9b54f4c50a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } } }
I20260807 19:38:14.065305 22158 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:38:14.066740 22157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "00cff960e6a84647bf857c9b54f4c50a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } } }
I20260807 19:38:14.067075 22157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:14.067526 22158 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "00cff960e6a84647bf857c9b54f4c50a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "128d4df4ad934462bd6508db5f4fe86f" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41075 } } peers { permanent_uuid: "00cff960e6a84647bf857c9b54f4c50a" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 38459 } attrs { promote: false } } peers { permanent_uuid: "d00a05c1214f47fc8dfaafec40e4f231" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 44121 } attrs { promote: false } } }
I20260807 19:38:14.067817 22158 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00a05c1214f47fc8dfaafec40e4f231 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:38:14.085659 22161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.075595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:35213 (local address 127.0.0.1:33720)
0807 19:38:14.075955 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.075970 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:14.076076 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:14.077408 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:14.077416 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:14.077453 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:14.077895 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.077911 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.079957 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.079966 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.082688 (+ 2722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.082710 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.084847 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.084853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.085011 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.085035 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:38:14.085148 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 19:38:14.085305 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253}
I20260807 19:38:14.086624 22178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.075935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.14.16.66:35213)
0807 19:38:14.076968 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:14.076977 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:14.077001 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:38:14.077090 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:14.077098 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:14.077198 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:14.078077 (+ 879us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:14.078294 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.079755 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.082971 (+ 3216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.084639 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.084784 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.084896 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:14.085281 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 19:38:14.086341 (+ 1060us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":286,"threads_started":1}
I20260807 19:38:14.087706 21872 tablet_service.cc:1467] Processing CreateTablet for tablet 786722fff2994375aee02edb36b1db4f (DEFAULT_TABLE table=test-table [id=03eb669f1989426db79c9daa7f5eef81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:14.088220 21872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 786722fff2994375aee02edb36b1db4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:14.090247 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.075144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:37913 (local address 127.0.0.1:48546)
0807 19:38:14.075394 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.075413 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:14.075525 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:14.085115 (+ 9590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:14.085122 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:14.085151 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:14.085325 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.085334 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.086762 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.086768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.088165 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.088177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.089338 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.089342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.089675 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.089714 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 19:38:14.089807 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 19:38:14.089942 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":90}
I20260807 19:38:14.090510 22180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.076786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54794 (local address 127.14.16.65:38289)
0807 19:38:14.083614 (+ 6828us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:14.083619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:14.083635 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:38:14.083693 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:14.083696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:14.083751 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:14.083874 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:14.084528 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.085504 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.087923 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.089093 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.089223 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.089329 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:14.089887 (+ 558us) server_negotiation.cc:299] Negotiation successful
0807 19:38:14.090029 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6717,"thread_start_us":6618,"threads_started":1}
I20260807 19:38:14.091085 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.075320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48546 (local address 127.14.16.67:37913)
0807 19:38:14.084802 (+ 9482us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:14.084808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:14.084825 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:38:14.084890 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:14.084895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:14.084975 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:14.085079 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:14.085500 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.086606 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.088391 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.089135 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.089410 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.089609 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:14.090557 (+ 948us) server_negotiation.cc:299] Negotiation successful
0807 19:38:14.090704 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9344,"thread_start_us":9236,"threads_started":1}
I20260807 19:38:14.091177 22179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.076395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:38289 (local address 127.0.0.1:54794)
0807 19:38:14.076914 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.076934 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:14.077040 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:14.083890 (+ 6850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:14.083894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:14.083922 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:14.084310 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.084322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.085653 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.085658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.087728 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.087745 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.089333 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.089340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.089580 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.089690 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 19:38:14.089815 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 19:38:14.090008 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":43,"thread_start_us":496,"threads_started":1}
I20260807 19:38:14.091928 21722 tablet_service.cc:1467] Processing CreateTablet for tablet 786722fff2994375aee02edb36b1db4f (DEFAULT_TABLE table=test-table [id=03eb669f1989426db79c9daa7f5eef81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:14.091982 22017 tablet_service.cc:1467] Processing CreateTablet for tablet 786722fff2994375aee02edb36b1db4f (DEFAULT_TABLE table=test-table [id=03eb669f1989426db79c9daa7f5eef81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:14.092356 21722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 786722fff2994375aee02edb36b1db4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:14.092437 22017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 786722fff2994375aee02edb36b1db4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:14.095105 22181 tablet_bootstrap.cc:492] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8: Bootstrap starting.
I20260807 19:38:14.097074 22181 tablet_bootstrap.cc:654] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:14.098456 22182 tablet_bootstrap.cc:492] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69: Bootstrap starting.
I20260807 19:38:14.100193 22182 tablet_bootstrap.cc:654] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:14.100332 22184 tablet_bootstrap.cc:492] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df: Bootstrap starting.
I20260807 19:38:14.101760 22181 tablet_bootstrap.cc:492] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8: No bootstrap required, opened a new log
I20260807 19:38:14.101918 22181 ts_tablet_manager.cc:1397] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260807 19:38:14.102500 22184 tablet_bootstrap.cc:654] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:14.102648 22181 raft_consensus.cc:348] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } } peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } }
I20260807 19:38:14.102888 22181 raft_consensus.cc:374] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:14.102993 22181 raft_consensus.cc:729] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7ee3db77e843c982f7c410345536f8, State: Initialized, Role: FOLLOWER
I20260807 19:38:14.103242 22181 consensus_queue.cc:260] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } } peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } }
I20260807 19:38:14.107095 22181 ts_tablet_manager.cc:1428] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260807 19:38:14.109722 22184 tablet_bootstrap.cc:492] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df: No bootstrap required, opened a new log
I20260807 19:38:14.109921 22184 ts_tablet_manager.cc:1397] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 19:38:14.110929 22184 raft_consensus.cc:348] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } } peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } }
I20260807 19:38:14.111129 22184 raft_consensus.cc:374] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:14.111243 22184 raft_consensus.cc:729] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adcee9bcb55c4940b2cbce43184628df, State: Initialized, Role: FOLLOWER
I20260807 19:38:14.111508 22184 consensus_queue.cc:260] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } } peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } }
I20260807 19:38:14.113454 22184 ts_tablet_manager.cc:1428] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 19:38:14.115022 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.114087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.65:45027)
0807 19:38:14.114612 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.114779 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"threads_started":1}
I20260807 19:38:14.117501 22182 tablet_bootstrap.cc:492] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69: No bootstrap required, opened a new log
I20260807 19:38:14.117669 22182 ts_tablet_manager.cc:1397] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.007s
I20260807 19:38:14.117719 22188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.107716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.66:55451)
0807 19:38:14.116546 (+ 8830us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.117252 (+ 706us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8691,"thread_start_us":8545,"threads_started":1}
I20260807 19:38:14.117705 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.117210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.65:35899)
0807 19:38:14.117436 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.117519 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 19:38:14.118706 22188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.118360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.66:41251)
0807 19:38:14.118544 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.118607 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 19:38:14.118547 22182 raft_consensus.cc:348] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } } peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } }
I20260807 19:38:14.118804 22182 raft_consensus.cc:374] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:14.118921 22182 raft_consensus.cc:729] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b170bb276e7c4f1cbe39f4c8208f7e69, State: Initialized, Role: FOLLOWER
I20260807 19:38:14.119166 21941 heartbeater.cc:503] Master 127.14.16.125:38459 was elected leader, sending a full tablet report...
I20260807 19:38:14.119323 21791 heartbeater.cc:503] Master 127.14.16.125:38459 was elected leader, sending a full tablet report...
I20260807 19:38:14.119176 22182 consensus_queue.cc:260] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } } peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } }
I20260807 19:38:14.120026 22182 ts_tablet_manager.cc:1428] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:38:14.122519 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.121208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.67:46231)
0807 19:38:14.122106 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.122274 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":771,"thread_start_us":577,"threads_started":1}
I20260807 19:38:14.123555 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.123112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.67:52695)
0807 19:38:14.123341 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.123418 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 19:38:14.123641 22085 heartbeater.cc:503] Master 127.14.16.125:38459 was elected leader, sending a full tablet report...
I20260807 19:38:14.124272 22186 raft_consensus.cc:482] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:38:14.124471 22186 raft_consensus.cc:504] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } } peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } }
I20260807 19:38:14.126050 22186 leader_election.cc:288] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b170bb276e7c4f1cbe39f4c8208f7e69 (127.14.16.67:37913), adcee9bcb55c4940b2cbce43184628df (127.14.16.65:38289)
I20260807 19:38:14.132970 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.126364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:40309 (local address 127.14.16.67:37913)
0807 19:38:14.126545 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:14.126551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:14.126572 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:38:14.126759 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:14.126767 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:14.126861 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:14.126982 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:14.127586 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.128587 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.131018 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.132148 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.132320 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.132394 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:14.132605 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 19:38:14.132749 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:38:14.133046 22188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.126307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:37913 (local address 127.14.16.66:40309)
0807 19:38:14.126458 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.126477 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:14.126619 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:14.127026 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:14.127032 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:14.127067 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:14.127444 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.127460 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.128714 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.128718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.130855 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.130871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.132318 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.132322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.132442 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.132465 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:38:14.132579 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 19:38:14.132733 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":42}
I20260807 19:38:14.133611 22180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.126725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:55725 (local address 127.14.16.65:38289)
0807 19:38:14.126892 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:14.126895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:14.127494 (+ 599us) server_negotiation.cc:407] Connection header received
0807 19:38:14.127636 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:14.127644 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:14.127716 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:14.127821 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:14.128250 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.129076 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.131404 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.132625 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.132780 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.132851 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:14.133275 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 19:38:14.133423 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:38:14.133682 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.126653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:38289 (local address 127.14.16.66:55725)
0807 19:38:14.127386 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.127410 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:14.127540 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:14.127839 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:14.127841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:14.127860 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:14.128117 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.128123 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.129203 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.129208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.131219 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:14.131229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.132845 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.132851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:14.133050 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.133089 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 19:38:14.133231 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 19:38:14.133322 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":40,"thread_start_us":415,"threads_started":1}
I20260807 19:38:14.133863 22038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786722fff2994375aee02edb36b1db4f" candidate_uuid: "bf7ee3db77e843c982f7c410345536f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" is_pre_election: true
I20260807 19:38:14.134109 22038 raft_consensus.cc:2454] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf7ee3db77e843c982f7c410345536f8 in term 0.
I20260807 19:38:14.134454 21743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786722fff2994375aee02edb36b1db4f" candidate_uuid: "bf7ee3db77e843c982f7c410345536f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adcee9bcb55c4940b2cbce43184628df" is_pre_election: true
I20260807 19:38:14.134655 21826 leader_election.cc:302] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b170bb276e7c4f1cbe39f4c8208f7e69, bf7ee3db77e843c982f7c410345536f8; no voters:
I20260807 19:38:14.134828 21743 raft_consensus.cc:2454] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf7ee3db77e843c982f7c410345536f8 in term 0.
I20260807 19:38:14.135061 22186 raft_consensus.cc:2781] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:38:14.135182 22186 raft_consensus.cc:482] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:38:14.135340 22186 raft_consensus.cc:3037] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:14.137790 22186 raft_consensus.cc:504] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } } peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } }
I20260807 19:38:14.138338 22186 leader_election.cc:288] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [CANDIDATE]: Term 1 election: Requested vote from peers b170bb276e7c4f1cbe39f4c8208f7e69 (127.14.16.67:37913), adcee9bcb55c4940b2cbce43184628df (127.14.16.65:38289)
I20260807 19:38:14.138796 22038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786722fff2994375aee02edb36b1db4f" candidate_uuid: "bf7ee3db77e843c982f7c410345536f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69"
I20260807 19:38:14.138895 21743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786722fff2994375aee02edb36b1db4f" candidate_uuid: "bf7ee3db77e843c982f7c410345536f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adcee9bcb55c4940b2cbce43184628df"
I20260807 19:38:14.139005 22038 raft_consensus.cc:3037] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:14.139068 21743 raft_consensus.cc:3037] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:14.142014 22038 raft_consensus.cc:2454] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf7ee3db77e843c982f7c410345536f8 in term 1.
I20260807 19:38:14.142025 21743 raft_consensus.cc:2454] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf7ee3db77e843c982f7c410345536f8 in term 1.
I20260807 19:38:14.142367 21826 leader_election.cc:302] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adcee9bcb55c4940b2cbce43184628df, bf7ee3db77e843c982f7c410345536f8; no voters:
I20260807 19:38:14.142648 22186 raft_consensus.cc:2781] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:14.143085 22186 raft_consensus.cc:686] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 1 LEADER]: Becoming Leader. State: Replica: bf7ee3db77e843c982f7c410345536f8, State: Running, Role: LEADER
I20260807 19:38:14.143498 22186 consensus_queue.cc:237] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } } peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } }
I20260807 19:38:14.144336 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.143928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.66:55479)
0807 19:38:14.144070 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.144170 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 19:38:14.146739 21408 catalog_manager.cc:5495] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf7ee3db77e843c982f7c410345536f8 (127.14.16.66). New cstate: current_term: 1 leader_uuid: "bf7ee3db77e843c982f7c410345536f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf7ee3db77e843c982f7c410345536f8" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 } health_report { overall_health: UNKNOWN } } }
I20260807 19:38:14.163079 22178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.158359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33726 (local address 127.14.16.66:35213)
0807 19:38:14.158544 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:14.158549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:14.158617 (+ 68us) server_negotiation.cc:407] Connection header received
0807 19:38:14.158835 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:14.158839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:14.158911 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:14.159029 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:38:14.159377 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.160087 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.161316 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.161693 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.161779 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.162138 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:38:14.162638 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:38:14.162739 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:14.162829 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 19:38:14.162888 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 19:38:14.167091 22038 raft_consensus.cc:1261] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69 [term 1 FOLLOWER]: Refusing update from remote peer bf7ee3db77e843c982f7c410345536f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:14.167419 21743 raft_consensus.cc:1261] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df [term 1 FOLLOWER]: Refusing update from remote peer bf7ee3db77e843c982f7c410345536f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:14.167821 22194 consensus_queue.cc:1035] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b170bb276e7c4f1cbe39f4c8208f7e69" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:14.168182 22186 consensus_queue.cc:1035] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adcee9bcb55c4940b2cbce43184628df" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 38289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:14.170553 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.170089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.67:36227)
0807 19:38:14.170290 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.170396 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 19:38:14.175805 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.171923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.65:38617)
0807 19:38:14.175475 (+ 3552us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.175582 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3406}
I20260807 19:38:14.182550 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.181995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.65:40667)
0807 19:38:14.182285 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.182390 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
I20260807 19:38:14.192096 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.191485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.67:50903)
0807 19:38:14.191758 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.191870 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260807 19:38:14.197057 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.195570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.66:50321)
0807 19:38:14.196704 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.196825 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":975}
I20260807 19:38:14.201138 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.200563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.14.16.66:52563)
0807 19:38:14.200887 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.200971 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
I20260807 19:38:14.209678 21407 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:14.209945 21407 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:14.215982 21407 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00cff960e6a84647bf857c9b54f4c50a: Sending DeleteTablet for 3 replicas of tablet 786722fff2994375aee02edb36b1db4f
I20260807 19:38:14.216945 22017 tablet_service.cc:1514] Processing DeleteTablet for tablet 786722fff2994375aee02edb36b1db4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:14 UTC) from {username='oryx', principal='oryx/127.14.16.125@KRBTEST.COM'} at 127.0.0.1:48546
I20260807 19:38:14.217093 21872 tablet_service.cc:1514] Processing DeleteTablet for tablet 786722fff2994375aee02edb36b1db4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:14 UTC) from {username='oryx', principal='oryx/127.14.16.125@KRBTEST.COM'} at 127.0.0.1:33720
I20260807 19:38:14.217876 21722 tablet_service.cc:1514] Processing DeleteTablet for tablet 786722fff2994375aee02edb36b1db4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:14 UTC) from {username='oryx', principal='oryx/127.14.16.125@KRBTEST.COM'} at 127.0.0.1:54794
I20260807 19:38:14.218921 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 21591
I20260807 19:38:14.228823 22209 tablet_replica.cc:330] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df: stopping tablet replica
I20260807 19:38:14.229205 22209 raft_consensus.cc:2229] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:14.229647 22209 raft_consensus.cc:2258] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:14.231076 22209 ts_tablet_manager.cc:1905] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:14.231742 22207 tablet_replica.cc:330] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69: stopping tablet replica
I20260807 19:38:14.231946 22207 raft_consensus.cc:2229] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:14.232252 22207 raft_consensus.cc:2258] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:14.233848 22207 ts_tablet_manager.cc:1905] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:14.236891 22209 ts_tablet_manager.cc:1918] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:14.237015 22209 log.cc:1196] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/786722fff2994375aee02edb36b1db4f
I20260807 19:38:14.237423 22209 ts_tablet_manager.cc:1939] T 786722fff2994375aee02edb36b1db4f P adcee9bcb55c4940b2cbce43184628df: Deleting consensus metadata
I20260807 19:38:14.238479 22208 tablet_replica.cc:330] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8: stopping tablet replica
I20260807 19:38:14.238515 21395 catalog_manager.cc:4841] TS adcee9bcb55c4940b2cbce43184628df (127.14.16.65:38289): tablet 786722fff2994375aee02edb36b1db4f (table test-table [id=03eb669f1989426db79c9daa7f5eef81]) successfully deleted
I20260807 19:38:14.238694 22208 raft_consensus.cc:2229] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:38:14.239214 22208 raft_consensus.cc:2258] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:14.240108 22208 ts_tablet_manager.cc:1905] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:14.242623 22207 ts_tablet_manager.cc:1918] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:14.243556 22207 log.cc:1196] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/786722fff2994375aee02edb36b1db4f
I20260807 19:38:14.244081 22207 ts_tablet_manager.cc:1939] T 786722fff2994375aee02edb36b1db4f P b170bb276e7c4f1cbe39f4c8208f7e69: Deleting consensus metadata
I20260807 19:38:14.246304 21395 catalog_manager.cc:4841] TS b170bb276e7c4f1cbe39f4c8208f7e69 (127.14.16.67:37913): tablet 786722fff2994375aee02edb36b1db4f (table test-table [id=03eb669f1989426db79c9daa7f5eef81]) successfully deleted
I20260807 19:38:14.246475 22208 ts_tablet_manager.cc:1918] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:14.246608 22208 log.cc:1196] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/786722fff2994375aee02edb36b1db4f
I20260807 19:38:14.247051 22208 ts_tablet_manager.cc:1939] T 786722fff2994375aee02edb36b1db4f P bf7ee3db77e843c982f7c410345536f8: Deleting consensus metadata
I20260807 19:38:14.248046 21393 catalog_manager.cc:4841] TS bf7ee3db77e843c982f7c410345536f8 (127.14.16.66:35213): tablet 786722fff2994375aee02edb36b1db4f (table test-table [id=03eb669f1989426db79c9daa7f5eef81]) successfully deleted
I20260807 19:38:14.254366 22180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.219804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.14.16.65:38289)
0807 19:38:14.219987 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:14.219995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:14.233123 (+ 13128us) server_negotiation.cc:407] Connection header received
0807 19:38:14.233321 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:14.233326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:14.233409 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:14.233511 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:14.241451 (+ 7940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.242562 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.244401 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.245027 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.245165 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.248741 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:14.248754 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:14.248774 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:14.248813 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:14.251649 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:14.252436 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:14.252442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:14.252445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:14.252565 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:14.252966 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:14.252969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:14.252971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:14.253312 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:14.253569 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:14.253876 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 19:38:14.254041 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:38:14.255044 21783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:14.377499 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 21591
I20260807 19:38:14.412442 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 21802
I20260807 19:38:14.427587 22178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.413201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33738 (local address 127.14.16.66:35213)
0807 19:38:14.413413 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:14.413420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:14.413471 (+ 51us) server_negotiation.cc:407] Connection header received
0807 19:38:14.413666 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:14.413671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:14.413771 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:14.413893 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:14.415340 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.416464 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.417677 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.418396 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.418544 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.420999 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:14.421011 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:14.421028 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:14.421070 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:14.424772 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:14.425431 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:14.425444 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:14.425446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:14.425582 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:14.425966 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:14.425972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:14.425975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:14.426326 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:14.426689 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:14.426958 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 19:38:14.427165 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 19:38:14.428344 21933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:14.555922 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 21802
I20260807 19:38:14.578294 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 21952
I20260807 19:38:14.592494 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.579084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48560 (local address 127.14.16.67:37913)
0807 19:38:14.579262 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:14.579267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:14.579322 (+ 55us) server_negotiation.cc:407] Connection header received
0807 19:38:14.579581 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:14.579587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:14.579693 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:14.579794 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:14.581171 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.582016 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.583234 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:14.583773 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:14.583880 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:14.586289 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:14.586304 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:14.586321 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:14.586354 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:14.589433 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:14.590042 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:14.590050 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:14.590052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:14.590195 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:14.590617 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:14.590622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:14.590625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:14.591107 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:14.591462 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:14.591810 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 19:38:14.592100 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 19:38:14.593117 22078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:14.637054 22214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.636255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54810)
0807 19:38:14.636778 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.636891 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":295,"threads_started":1}
I20260807 19:38:14.638350 22214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:14.638017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41075 (local address 127.0.0.1:54818)
0807 19:38:14.638167 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:14.638244 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:41075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 19:38:14.687646 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 21952
I20260807 19:38:14.707695 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 21267
I20260807 19:38:14.708825 21438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:14.835069 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 21267
I20260807 19:38:14.859268 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 21369
I20260807 19:38:14.860462 21632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:14.978679 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 21369
2026-08-07T19:38:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9543 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:38:15 dist-test-slave-07c2 krb5kdc[22220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:38:15 dist-test-slave-07c2 krb5kdc[22220](info): set up 2 sockets
Aug 07 19:38:15 dist-test-slave-07c2 krb5kdc[22220](info): commencing operation
krb5kdc: starting...
W20260807 19:38:16.107141 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:38:16 dist-test-slave-07c2 krb5kdc[22220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:38:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:38:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.126.keytab.
Entry for principal oryx/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.126.keytab.
I20260807 19:38:16.355784 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:39419
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.126.keytab
--principal=oryx/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:39419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:16.684373 22233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:16.684666 22233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:16.684837 22233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:16.692003 22233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:38:16.692123 22233 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 19:38:16.692198 22233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:16.692309 22233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:16.692391 22233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:38:16.692483 22233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:38:16.702342 22233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:39419
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.126.keytab
--principal=oryx/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:39419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:16.703606 22233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:16.704852 22233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:16.716322 22238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:16.716536 22239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:16.717381 22241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:16.718156 22233 server_base.cc:1034] running on GCE node
I20260807 19:38:16.718851 22233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:16.720184 22233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:16.721400 22233 hybrid_clock.cc:648] HybridClock initialized: now 1786131496721335 us; error 50 us; skew 500 ppm
I20260807 19:38:16.721728 22233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:16 dist-test-slave-07c2 krb5kdc[22220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131496, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:16.724785 22233 init.cc:377] Logged in from keytab as oryx/127.14.16.126@KRBTEST.COM (short username oryx)
I20260807 19:38:16.726177 22233 webserver.cc:460] Webserver started at http://127.14.16.126:44055/ using document root <none> and password file <none>
I20260807 19:38:16.726908 22233 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:16.727070 22233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:16.727476 22233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:16.730331 22233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "2ad8a40fa4ff4d0fbf9cf26f900cd723"
format_stamp: "Formatted at 2026-08-07 19:38:16 on dist-test-slave-07c2"
server_key: "50dd8cd67916accbdb1290613ce84bb9"
server_key_iv: "8f7e9b410a957d5951eeaad8b5dcd6d6"
server_key_version: "encryptionkey@0"
I20260807 19:38:16.731117 22233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "2ad8a40fa4ff4d0fbf9cf26f900cd723"
format_stamp: "Formatted at 2026-08-07 19:38:16 on dist-test-slave-07c2"
server_key: "50dd8cd67916accbdb1290613ce84bb9"
server_key_iv: "8f7e9b410a957d5951eeaad8b5dcd6d6"
server_key_version: "encryptionkey@0"
I20260807 19:38:16.735955 22233 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:38:16.739251 22248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:16.740532 22233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:38:16.740717 22233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "2ad8a40fa4ff4d0fbf9cf26f900cd723"
format_stamp: "Formatted at 2026-08-07 19:38:16 on dist-test-slave-07c2"
server_key: "50dd8cd67916accbdb1290613ce84bb9"
server_key_iv: "8f7e9b410a957d5951eeaad8b5dcd6d6"
server_key_version: "encryptionkey@0"
I20260807 19:38:16.740939 22233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:16.756317 22233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:16.760038 22233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:16.760308 22233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:16.781644 22233 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:39419
I20260807 19:38:16.781666 22312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:39419 every 8 connection(s)
I20260807 19:38:16.783035 22233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:38:16.786778 22313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:16.790215 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 22233
I20260807 19:38:16.790427 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:38:16.791239 14401 external_mini_cluster.cc:1428] Setting key 7af7a6fc533c86e1f138ba4b16c26193
I20260807 19:38:16.799201 22313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723: Bootstrap starting.
I20260807 19:38:16.802313 22313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:16.803339 22313 log.cc:824] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:38:16 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:16.806066 22313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723: No bootstrap required, opened a new log
I20260807 19:38:16.813133 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:16.793379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43954 (local address 127.14.16.126:39419)
0807 19:38:16.794162 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:16.794173 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:16.794212 (+ 39us) server_negotiation.cc:407] Connection header received
0807 19:38:16.795964 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:16.795991 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:16.796365 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:16.796768 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:16.798194 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:16.799557 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:16.800833 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:16.801483 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:16.801682 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:16.805711 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:16.805720 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:16.805740 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:16.805775 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:16.808923 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:16.809635 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:16.809642 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:16.809648 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:16.809817 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:16.810207 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:16.810214 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:16.810225 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:16.810849 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:16.811168 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:16.811681 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 19:38:16.812114 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":381,"threads_started":1}
I20260807 19:38:16.814234 22313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad8a40fa4ff4d0fbf9cf26f900cd723" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39419 } }
I20260807 19:38:16.814723 22313 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:16.814838 22313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad8a40fa4ff4d0fbf9cf26f900cd723, State: Initialized, Role: FOLLOWER
I20260807 19:38:16.815546 22313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad8a40fa4ff4d0fbf9cf26f900cd723" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39419 } }
I20260807 19:38:16.815820 22313 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:38:16.815994 22313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:38:16.816195 22313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:16.818384 22313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad8a40fa4ff4d0fbf9cf26f900cd723" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39419 } }
I20260807 19:38:16.818831 22313 leader_election.cc:302] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ad8a40fa4ff4d0fbf9cf26f900cd723; no voters:
I20260807 19:38:16.819475 22313 leader_election.cc:288] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:38:16.819566 22318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:16.820351 22318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [term 1 LEADER]: Becoming Leader. State: Replica: 2ad8a40fa4ff4d0fbf9cf26f900cd723, State: Running, Role: LEADER
I20260807 19:38:16.820911 22318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad8a40fa4ff4d0fbf9cf26f900cd723" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39419 } }
I20260807 19:38:16.821612 22313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:38:16.824383 22320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ad8a40fa4ff4d0fbf9cf26f900cd723. Latest consensus state: current_term: 1 leader_uuid: "2ad8a40fa4ff4d0fbf9cf26f900cd723" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad8a40fa4ff4d0fbf9cf26f900cd723" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39419 } } }
I20260807 19:38:16.824862 22320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:16.825721 22319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ad8a40fa4ff4d0fbf9cf26f900cd723" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad8a40fa4ff4d0fbf9cf26f900cd723" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39419 } } }
I20260807 19:38:16.826061 22319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:16.830273 22324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:38:16.835556 22324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:38:16.845208 22324 catalog_manager.cc:1347] Generated new cluster ID: 59dd661b16314067bb1ba85197edef56
I20260807 19:38:16.845448 22324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:38:16.857591 22324 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:38:16.858912 22324 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:38:16.867846 22324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723: Generated new TSK 0
I20260807 19:38:16.868677 22324 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab.
Entry for principal oryx/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab.
I20260807 19:38:16.934667 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39419
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab
--principal=oryx/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:17.270325 22341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:17.270605 22341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:17.270702 22341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:17.279079 22341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:17.279186 22341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:17.279273 22341 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:38:17.279429 22341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:38:17.289064 22341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab
--principal=oryx/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.16.126:39419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:17.290318 22341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:17.291740 22341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:17.304730 22346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:17.304808 22347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:17.304926 22349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:17.305528 22341 server_base.cc:1034] running on GCE node
I20260807 19:38:17.306183 22341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:17.307108 22341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:17.308326 22341 hybrid_clock.cc:648] HybridClock initialized: now 1786131497308278 us; error 58 us; skew 500 ppm
I20260807 19:38:17.308640 22341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:17 dist-test-slave-07c2 krb5kdc[22220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131497, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:17.312086 22341 init.cc:377] Logged in from keytab as oryx/127.14.16.65@KRBTEST.COM (short username oryx)
I20260807 19:38:17.313556 22341 webserver.cc:460] Webserver started at http://127.14.16.65:39345/ using document root <none> and password file <none>
I20260807 19:38:17.314242 22341 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:17.314361 22341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:17.314718 22341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:17.317647 22341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "64d72e7db6f44857b2902f8bec7a4810"
format_stamp: "Formatted at 2026-08-07 19:38:17 on dist-test-slave-07c2"
server_key: "596f487dde22846e794d888ccdec34c8"
server_key_iv: "7d9781bee77705cf59dc61999b5bf7ec"
server_key_version: "encryptionkey@0"
I20260807 19:38:17.318445 22341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "64d72e7db6f44857b2902f8bec7a4810"
format_stamp: "Formatted at 2026-08-07 19:38:17 on dist-test-slave-07c2"
server_key: "596f487dde22846e794d888ccdec34c8"
server_key_iv: "7d9781bee77705cf59dc61999b5bf7ec"
server_key_version: "encryptionkey@0"
I20260807 19:38:17.323632 22341 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 19:38:17.326959 22356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:17.328255 22341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:38:17.328418 22341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "64d72e7db6f44857b2902f8bec7a4810"
format_stamp: "Formatted at 2026-08-07 19:38:17 on dist-test-slave-07c2"
server_key: "596f487dde22846e794d888ccdec34c8"
server_key_iv: "7d9781bee77705cf59dc61999b5bf7ec"
server_key_version: "encryptionkey@0"
I20260807 19:38:17.328662 22341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:17.356268 22341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:17.360390 22341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:17.360668 22341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:17.363505 22341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:17.363607 22341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:17.363783 22341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:17.363842 22341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:38:17 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131497, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:17.390393 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:17.364335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:55875 (local address 127.14.16.126:39419)
0807 19:38:17.364853 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:17.364861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:17.366217 (+ 1356us) server_negotiation.cc:407] Connection header received
0807 19:38:17.368992 (+ 2775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:17.369000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:17.369101 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:17.369173 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:17.372438 (+ 3265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.373731 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.376507 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.377164 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.377322 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:17.382840 (+ 5518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:17.382849 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:17.382878 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:17.382934 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:17.386747 (+ 3813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:17.387687 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:17.387693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:17.387697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:17.387843 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:17.388351 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:17.388357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:17.388361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:17.388672 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:17.388858 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:17.389718 (+ 860us) server_negotiation.cc:299] Negotiation successful
0807 19:38:17.389962 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":278,"threads_started":1}
I20260807 19:38:17.391786 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:17.365048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39419 (local address 127.14.16.65:55875)
0807 19:38:17.366008 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:17.366084 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:17.368596 (+ 2512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:17.369848 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:17.369863 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:17.371011 (+ 1148us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:17.372111 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:17.372140 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.373929 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.373938 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:17.376247 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:17.376269 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.377446 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.377459 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:17.377740 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:17.378669 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:17.378713 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:17.382595 (+ 3882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:17.386901 (+ 4306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:17.386910 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:17.386928 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:17.387526 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:17.387970 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:17.387976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:17.387980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:17.388214 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:17.388837 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:17.388845 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:17.389267 (+ 422us) client_negotiation.cc:769] Sending connection context
0807 19:38:17.389589 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 19:38:17.390162 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":428,"threads_started":1}
I20260807 19:38:17.414310 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:17.397696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:55819 (local address 127.14.16.126:39419)
0807 19:38:17.397909 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:17.397918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:17.398764 (+ 846us) server_negotiation.cc:407] Connection header received
0807 19:38:17.398990 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:17.398995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:17.399065 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:17.399165 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:17.400578 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.401876 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.404256 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.405015 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.405188 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:17.407460 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:17.407469 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:17.407477 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:17.407527 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:17.411005 (+ 3478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:17.411941 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:17.411946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:17.411950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:17.412113 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:17.412625 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:17.412630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:17.412633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:17.412945 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:17.413132 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:17.413689 (+ 557us) server_negotiation.cc:299] Negotiation successful
0807 19:38:17.413913 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:38:17.414521 22432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:17.397698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39419 (local address 127.14.16.65:55819)
0807 19:38:17.398653 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:17.398675 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:17.398849 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:17.399191 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:17.399195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:17.399957 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:17.400397 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:17.400415 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.402054 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.402063 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:17.404068 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:17.404098 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.405282 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.405290 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:17.405465 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:17.406153 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:17.406171 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:17.407206 (+ 1035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:17.411148 (+ 3942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:17.411153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:17.411157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:17.411816 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:17.412257 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:17.412265 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:17.412269 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:17.412471 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:17.413213 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:17.413223 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:17.413427 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 19:38:17.413575 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 19:38:17.413892 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"thread_start_us":650,"threads_started":1}
I20260807 19:38:17.421700 22341 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:37685
I20260807 19:38:17.421720 22478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:37685 every 8 connection(s)
I20260807 19:38:17.423015 22341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:38:17.425066 22479 heartbeater.cc:347] Connected to a master server at 127.14.16.126:39419
I20260807 19:38:17.425297 22479 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:17.425917 22479 heartbeater.cc:511] Master 127.14.16.126:39419 requested a full tablet report, sending...
I20260807 19:38:17.427784 22265 ts_manager.cc:194] Registered new tserver with Master: 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685)
I20260807 19:38:17.429332 22265 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:55875
I20260807 19:38:17.430424 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 22341
I20260807 19:38:17.430649 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:38:17.431354 14401 external_mini_cluster.cc:1428] Setting key 73456257f408ae445367a2a6e7c61ee2
WARNING: no policy specified for oryx/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab.
Entry for principal oryx/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab.
I20260807 19:38:17.484097 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39419
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab
--principal=oryx/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:17.822373 22483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:17.822636 22483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:17.822824 22483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:17.830101 22483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:17.830232 22483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:17.830384 22483 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:38:17.830506 22483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:38:17.840440 22483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab
--principal=oryx/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.16.126:39419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:17.841668 22483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:17.843010 22483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:17.856158 22490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:17.856243 22489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:17.856235 22492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:17.857162 22483 server_base.cc:1034] running on GCE node
I20260807 19:38:17.857728 22483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:17.858584 22483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:17.859797 22483 hybrid_clock.cc:648] HybridClock initialized: now 1786131497859742 us; error 34 us; skew 500 ppm
I20260807 19:38:17.860087 22483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:17 dist-test-slave-07c2 krb5kdc[22220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131497, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:17.863158 22483 init.cc:377] Logged in from keytab as oryx/127.14.16.66@KRBTEST.COM (short username oryx)
I20260807 19:38:17.864671 22483 webserver.cc:460] Webserver started at http://127.14.16.66:37337/ using document root <none> and password file <none>
I20260807 19:38:17.865289 22483 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:17.865374 22483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:17.865650 22483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:17.868655 22483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "280e6bd8c951495ba5f75e6f9afe3b97"
format_stamp: "Formatted at 2026-08-07 19:38:17 on dist-test-slave-07c2"
server_key: "a7c97f7b4253326a198e8863c302c09c"
server_key_iv: "140f3b8e49b4f9617374f0c2b6f42d67"
server_key_version: "encryptionkey@0"
I20260807 19:38:17.869383 22483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "280e6bd8c951495ba5f75e6f9afe3b97"
format_stamp: "Formatted at 2026-08-07 19:38:17 on dist-test-slave-07c2"
server_key: "a7c97f7b4253326a198e8863c302c09c"
server_key_iv: "140f3b8e49b4f9617374f0c2b6f42d67"
server_key_version: "encryptionkey@0"
I20260807 19:38:17.874344 22483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 19:38:17.877507 22500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:17.878697 22483 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 19:38:17.878858 22483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "280e6bd8c951495ba5f75e6f9afe3b97"
format_stamp: "Formatted at 2026-08-07 19:38:17 on dist-test-slave-07c2"
server_key: "a7c97f7b4253326a198e8863c302c09c"
server_key_iv: "140f3b8e49b4f9617374f0c2b6f42d67"
server_key_version: "encryptionkey@0"
I20260807 19:38:17.879082 22483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:17.896834 22483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:17.901079 22483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:17.901373 22483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:17.904469 22483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:17.904569 22483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:17.904719 22483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:17.904824 22483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:38:17 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131497, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:17.930414 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:17.905650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39419 (local address 127.14.16.66:45867)
0807 19:38:17.906473 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:17.906517 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:17.908500 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:17.909252 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:17.909260 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:17.910006 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:17.911046 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:17.911077 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.912743 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.912755 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:17.914845 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:17.914867 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.915999 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.916008 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:17.916363 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:17.917247 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:17.917285 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:17.921285 (+ 4000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:17.925055 (+ 3770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:17.925066 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:17.925089 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:17.925987 (+ 898us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:17.926529 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:17.926533 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:17.926535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:17.926937 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:17.927689 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:17.927698 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:17.928362 (+ 664us) client_negotiation.cc:769] Sending connection context
0807 19:38:17.928776 (+ 414us) client_negotiation.cc:240] Negotiation successful
0807 19:38:17.929287 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":364,"threads_started":1}
I20260807 19:38:17.931525 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:17.905055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:45867 (local address 127.14.16.126:39419)
0807 19:38:17.905278 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:17.905286 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:17.906634 (+ 1348us) server_negotiation.cc:407] Connection header received
0807 19:38:17.908798 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:17.908804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:17.908897 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:17.909007 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:17.911324 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.912503 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.915070 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.915725 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.915881 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:17.921572 (+ 5691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:17.921581 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:17.921589 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:17.921636 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:17.924849 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:17.926141 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:17.926149 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:17.926154 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:17.926324 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:17.927057 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:17.927064 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:17.927068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:17.927530 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:17.927752 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:17.931003 (+ 3251us) server_negotiation.cc:299] Negotiation successful
0807 19:38:17.931190 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:38:17.953855 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:17.938410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:57497 (local address 127.14.16.126:39419)
0807 19:38:17.938637 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:17.938644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:17.939005 (+ 361us) server_negotiation.cc:407] Connection header received
0807 19:38:17.939235 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:17.939239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:17.939362 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:17.939487 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:17.940972 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.942060 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.944150 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.944684 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.944826 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:17.946999 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:17.947010 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:17.947014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:17.947057 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:17.950493 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:17.951462 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:17.951468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:17.951472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:17.951670 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:17.952238 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:17.952245 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:17.952249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:17.952700 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:17.952827 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:17.953439 (+ 612us) server_negotiation.cc:299] Negotiation successful
0807 19:38:17.953605 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:38:17.954512 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:17.937853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39419 (local address 127.14.16.66:57497)
0807 19:38:17.938865 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:17.938894 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:17.939103 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:17.939523 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:17.939529 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:17.940263 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:17.940777 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:17.940794 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.942242 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.942248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:17.943998 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:17.944012 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:17.944904 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:17.944910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:17.945085 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:17.945993 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:17.946013 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:17.946805 (+ 792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:17.950642 (+ 3837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:17.950648 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:17.950653 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:17.951318 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:17.951842 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:17.951854 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:17.951860 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:17.952089 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:17.952844 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:17.952852 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:17.953141 (+ 289us) client_negotiation.cc:769] Sending connection context
0807 19:38:17.953338 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 19:38:17.953833 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":469,"threads_started":1}
I20260807 19:38:17.959066 22483 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:45805
I20260807 19:38:17.959089 22623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:45805 every 8 connection(s)
I20260807 19:38:17.960299 22483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:38:17.962248 22624 heartbeater.cc:347] Connected to a master server at 127.14.16.126:39419
I20260807 19:38:17.962445 22624 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:17.962999 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 22483
I20260807 19:38:17.963194 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:38:17.963232 22624 heartbeater.cc:511] Master 127.14.16.126:39419 requested a full tablet report, sending...
I20260807 19:38:17.963827 14401 external_mini_cluster.cc:1428] Setting key 8de355516879184033a4a249e928eab6
I20260807 19:38:17.964748 22265 ts_manager.cc:194] Registered new tserver with Master: 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66:45805)
I20260807 19:38:17.965727 22265 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:45867
WARNING: no policy specified for oryx/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab.
Entry for principal oryx/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab.
I20260807 19:38:18.016185 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39419
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab
--principal=oryx/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:18.346022 22628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:18.346339 22628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:18.346513 22628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:18.353684 22628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:18.353829 22628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:18.353976 22628 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:38:18.354123 22628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:38:18.363670 22628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab
--principal=oryx/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.16.126:39419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:18.364916 22628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:18.366189 22628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:18.379132 22635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:18.379347 22633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:18.379204 22637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:18.379982 22628 server_base.cc:1034] running on GCE node
I20260807 19:38:18.380570 22628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:18.381457 22628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:18.382715 22628 hybrid_clock.cc:648] HybridClock initialized: now 1786131498382646 us; error 54 us; skew 500 ppm
I20260807 19:38:18.383010 22628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:18 dist-test-slave-07c2 krb5kdc[22220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131498, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:18.386220 22628 init.cc:377] Logged in from keytab as oryx/127.14.16.67@KRBTEST.COM (short username oryx)
I20260807 19:38:18.387737 22628 webserver.cc:460] Webserver started at http://127.14.16.67:41897/ using document root <none> and password file <none>
I20260807 19:38:18.388376 22628 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:18.388461 22628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:18.388749 22628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:18.391636 22628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "d2af1c6e12ea424db07c1456c5dcdd5b"
format_stamp: "Formatted at 2026-08-07 19:38:18 on dist-test-slave-07c2"
server_key: "2da363369eb0a90192311bd353e636ee"
server_key_iv: "bcf32e0370524d0d4602107f5f9c51c7"
server_key_version: "encryptionkey@0"
I20260807 19:38:18.392376 22628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "d2af1c6e12ea424db07c1456c5dcdd5b"
format_stamp: "Formatted at 2026-08-07 19:38:18 on dist-test-slave-07c2"
server_key: "2da363369eb0a90192311bd353e636ee"
server_key_iv: "bcf32e0370524d0d4602107f5f9c51c7"
server_key_version: "encryptionkey@0"
I20260807 19:38:18.397406 22628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:38:18.400812 22644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:18.402110 22628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:38:18.402308 22628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "d2af1c6e12ea424db07c1456c5dcdd5b"
format_stamp: "Formatted at 2026-08-07 19:38:18 on dist-test-slave-07c2"
server_key: "2da363369eb0a90192311bd353e636ee"
server_key_iv: "bcf32e0370524d0d4602107f5f9c51c7"
server_key_version: "encryptionkey@0"
I20260807 19:38:18.402532 22628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:18.421792 22628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:18.427037 22628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:18.427364 22628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:18.430187 22628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:18.430302 22628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:18.430454 22628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:18.430526 22628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:18.432281 22479 heartbeater.cc:503] Master 127.14.16.126:39419 was elected leader, sending a full tablet report...
Aug 07 19:38:18 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131498, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:18.456833 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:18.431429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:59477 (local address 127.14.16.126:39419)
0807 19:38:18.431594 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:18.431599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:18.432794 (+ 1195us) server_negotiation.cc:407] Connection header received
0807 19:38:18.435696 (+ 2902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:18.435702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:18.435806 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:18.435914 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:18.438528 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:18.439540 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:18.442777 (+ 3237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:18.443355 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:18.443511 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:18.448907 (+ 5396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:18.448915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:18.448920 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:18.448964 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:18.452639 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:18.453797 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:18.453803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:18.453806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:18.453955 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:18.454608 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:18.454612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:18.454614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:18.454988 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:18.455141 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:18.456208 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0807 19:38:18.456450 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:38:18.457903 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:18.431529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39419 (local address 127.14.16.67:59477)
0807 19:38:18.432582 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:18.432681 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:18.435420 (+ 2739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:18.436241 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:18.436259 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:18.437253 (+ 994us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:18.438298 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:18.438329 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:18.439733 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:18.439738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:18.442602 (+ 2864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:18.442623 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:18.443619 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:18.443632 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:18.444039 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:18.444939 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:18.444975 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:18.448682 (+ 3707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:18.452832 (+ 4150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:18.452843 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:18.452864 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:18.453636 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:18.454087 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:18.454092 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:18.454096 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:18.454439 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:18.455237 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:18.455243 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:18.455741 (+ 498us) client_negotiation.cc:769] Sending connection context
0807 19:38:18.456101 (+ 360us) client_negotiation.cc:240] Negotiation successful
0807 19:38:18.456664 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":398,"threads_started":1}
I20260807 19:38:18.477957 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:18.463758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:60465 (local address 127.14.16.126:39419)
0807 19:38:18.463905 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:18.463909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:18.464642 (+ 733us) server_negotiation.cc:407] Connection header received
0807 19:38:18.464819 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:18.464824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:18.464902 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:18.465003 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:18.465949 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:18.466617 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:18.468270 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:18.468869 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:18.469027 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:18.471755 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:18.471760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:18.471763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:18.471792 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:18.475044 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:18.476006 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:18.476014 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:18.476017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:18.476146 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:18.476519 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:18.476524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:18.476528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:18.476801 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:18.476954 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:18.477368 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 19:38:18.477584 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 19:38:18.478060 22731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:18.463752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39419 (local address 127.14.16.67:60465)
0807 19:38:18.464545 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:18.464560 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:18.464706 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:18.465026 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:18.465030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:18.465530 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:18.465810 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:18.465819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:18.466773 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:18.466781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:18.468105 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:18.468120 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:18.469329 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:18.469338 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:18.469467 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:18.470291 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:18.470321 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:18.471568 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:18.475189 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:18.475195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:18.475198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:18.475877 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:18.476265 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:18.476270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:18.476273 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:18.476402 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:18.476917 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:18.476922 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:18.477106 (+ 184us) client_negotiation.cc:769] Sending connection context
0807 19:38:18.477241 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 19:38:18.477556 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":504,"threads_started":1}
I20260807 19:38:18.483872 22628 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:41783
I20260807 19:38:18.483913 22768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:41783 every 8 connection(s)
I20260807 19:38:18.485044 22628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:38:18.486959 22769 heartbeater.cc:347] Connected to a master server at 127.14.16.126:39419
I20260807 19:38:18.487157 22769 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:18.487815 22769 heartbeater.cc:511] Master 127.14.16.126:39419 requested a full tablet report, sending...
I20260807 19:38:18.489044 22265 ts_manager.cc:194] Registered new tserver with Master: d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783)
I20260807 19:38:18.489867 22265 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:59477
I20260807 19:38:18.491957 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 22628
I20260807 19:38:18.492129 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:38:18.492573 14401 external_mini_cluster.cc:1428] Setting key 0789491cb49a832bb81b31f979cc1cc4
I20260807 19:38:18.495406 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:38:18 dist-test-slave-07c2 krb5kdc[22220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131498, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:38:18 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:18 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:18.968050 22624 heartbeater.cc:503] Master 127.14.16.126:39419 was elected leader, sending a full tablet report...
I20260807 19:38:19.492295 22769 heartbeater.cc:503] Master 127.14.16.126:39419 was elected leader, sending a full tablet report...
W20260807 19:38:19.680610 22777 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.133s user 0.004s sys 0.015s
W20260807 19:38:19.681903 22365 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:18.541632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35688 (local address 127.14.16.126:39419)
0807 19:38:18.541813 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:18.541817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:18.542149 (+ 332us) server_negotiation.cc:407] Connection header received
0807 19:38:18.542325 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:18.542329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:18.542393 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:18.542488 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:18.543912 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:18.544699 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:18.546025 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:18.546442 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:18.546538 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:19.681595 (+1135057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35688: BlockingRecv error: recv got EOF from 127.0.0.1:35688 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 19:38:19 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131498, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:19.703927 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.685521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35702 (local address 127.14.16.126:39419)
0807 19:38:19.685768 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:19.685775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:19.686061 (+ 286us) server_negotiation.cc:407] Connection header received
0807 19:38:19.686286 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:19.686291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:19.686384 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:19.686499 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:19.687851 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.688634 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.690244 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.690718 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.690859 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:19.698853 (+ 7994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:19.698858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:19.698862 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:19.698900 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:19.701386 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:19.702010 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:19.702014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:19.702016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:19.702145 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:19.702476 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:19.702479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:19.702481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:19.702754 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:19.702979 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:19.703254 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 19:38:19.703518 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:38:19.708210 22265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:38:19.715287 22265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:38:19.749260 22791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.727450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45414 (local address 127.14.16.65:37685)
0807 19:38:19.740672 (+ 13222us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:19.740678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:19.740702 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:38:19.740780 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:19.740786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:19.741073 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:19.741282 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:19.742252 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.743898 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.746639 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.748049 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.748208 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:19.748415 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:19.748753 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 19:38:19.748922 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13075,"thread_start_us":12895,"threads_started":1}
I20260807 19:38:19.749349 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.727135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:37685 (local address 127.0.0.1:45414)
0807 19:38:19.728389 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:19.728436 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:19.728672 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:19.741564 (+ 12892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:19.741571 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:19.741607 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:19.742071 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:19.742082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.744050 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.744055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:19.746483 (+ 2428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:19.746495 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.748267 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.748272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:19.748467 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:19.748518 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 19:38:19.748705 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 19:38:19.748897 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":6,"spinlock_wait_cycles":4992,"thread_start_us":593,"threads_started":1}
I20260807 19:38:19.750726 22793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.727404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:41783 (local address 127.0.0.1:52226)
0807 19:38:19.742713 (+ 15309us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:19.742731 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:19.742852 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:19.743735 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:19.743743 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:19.743774 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:19.744266 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:19.744280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.746123 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.746128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:19.747937 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:19.747950 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.749849 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.749855 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:19.750034 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:19.750066 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:38:19.750156 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 19:38:19.750283 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15170,"mutex_wait_us":19,"thread_start_us":16126,"threads_started":1}
I20260807 19:38:19.750771 22792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.727616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52226 (local address 127.14.16.67:41783)
0807 19:38:19.739998 (+ 12382us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:19.740011 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:19.742852 (+ 2841us) server_negotiation.cc:407] Connection header received
0807 19:38:19.742987 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:19.742996 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:19.743226 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:19.743483 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:19.744477 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.745920 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.748096 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.749630 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.749782 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:19.749933 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:19.750222 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 19:38:19.750411 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12151,"thread_start_us":12402,"threads_started":1}
I20260807 19:38:19.753554 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.729358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:45805 (local address 127.0.0.1:44434)
0807 19:38:19.743783 (+ 14425us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:19.743799 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:19.743935 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:19.744506 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:19.744510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:19.744534 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:19.745055 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:19.745072 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.746905 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.746910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:19.749033 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:19.749051 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.752754 (+ 3703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.752760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:19.752939 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:19.752971 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:38:19.753075 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:38:19.753201 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14322,"mutex_wait_us":109,"thread_start_us":13060,"threads_started":1}
I20260807 19:38:19.753567 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.729615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44434 (local address 127.14.16.66:45805)
0807 19:38:19.743374 (+ 13759us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:19.743382 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:19.743913 (+ 531us) server_negotiation.cc:407] Connection header received
0807 19:38:19.744078 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:19.744087 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:19.744300 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:19.744501 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:19.745279 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.746766 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.749234 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.752524 (+ 3290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.752679 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:19.752876 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:19.753127 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 19:38:19.753285 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13575,"thread_start_us":13444,"threads_started":1}
I20260807 19:38:19.753404 22407 tablet_service.cc:1467] Processing CreateTablet for tablet 9f8e58de56894eef900d656fc69feea3 (DEFAULT_TABLE table=test-table [id=7f2f5b318e434d54849950e4d24da3dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:19.753978 22697 tablet_service.cc:1467] Processing CreateTablet for tablet 9f8e58de56894eef900d656fc69feea3 (DEFAULT_TABLE table=test-table [id=7f2f5b318e434d54849950e4d24da3dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:19.754839 22407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f8e58de56894eef900d656fc69feea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:19.755231 22697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f8e58de56894eef900d656fc69feea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:19.757817 22552 tablet_service.cc:1467] Processing CreateTablet for tablet 9f8e58de56894eef900d656fc69feea3 (DEFAULT_TABLE table=test-table [id=7f2f5b318e434d54849950e4d24da3dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:19.759091 22552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f8e58de56894eef900d656fc69feea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:19.774299 22797 tablet_bootstrap.cc:492] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97: Bootstrap starting.
I20260807 19:38:19.777696 22795 tablet_bootstrap.cc:492] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b: Bootstrap starting.
I20260807 19:38:19.778965 22797 tablet_bootstrap.cc:654] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:19.779808 22796 tablet_bootstrap.cc:492] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810: Bootstrap starting.
I20260807 19:38:19.780471 22797 log.cc:824] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:19.781188 22795 tablet_bootstrap.cc:654] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:19.782490 22795 log.cc:824] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:19.783663 22796 tablet_bootstrap.cc:654] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:19.784927 22796 log.cc:824] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:19.786842 22795 tablet_bootstrap.cc:492] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b: No bootstrap required, opened a new log
I20260807 19:38:19.787194 22795 ts_tablet_manager.cc:1397] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 19:38:19.793190 22797 tablet_bootstrap.cc:492] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97: No bootstrap required, opened a new log
I20260807 19:38:19.793540 22797 ts_tablet_manager.cc:1397] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.000s
I20260807 19:38:19.806277 22796 tablet_bootstrap.cc:492] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810: No bootstrap required, opened a new log
I20260807 19:38:19.806706 22796 ts_tablet_manager.cc:1397] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.015s
I20260807 19:38:19.806438 22797 raft_consensus.cc:348] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } }
I20260807 19:38:19.806912 22797 raft_consensus.cc:374] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:19.807040 22797 raft_consensus.cc:729] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280e6bd8c951495ba5f75e6f9afe3b97, State: Initialized, Role: FOLLOWER
I20260807 19:38:19.807734 22797 consensus_queue.cc:260] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } }
I20260807 19:38:19.808782 22795 raft_consensus.cc:348] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } }
I20260807 19:38:19.809291 22795 raft_consensus.cc:374] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:19.809429 22795 raft_consensus.cc:729] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2af1c6e12ea424db07c1456c5dcdd5b, State: Initialized, Role: FOLLOWER
I20260807 19:38:19.809554 22797 ts_tablet_manager.cc:1428] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97: Time spent starting tablet: real 0.016s user 0.012s sys 0.000s
I20260807 19:38:19.810627 22795 consensus_queue.cc:260] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } }
I20260807 19:38:19.812427 22795 ts_tablet_manager.cc:1428] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b: Time spent starting tablet: real 0.025s user 0.004s sys 0.012s
I20260807 19:38:19.816944 22796 raft_consensus.cc:348] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } }
I20260807 19:38:19.817385 22796 raft_consensus.cc:374] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:19.817478 22796 raft_consensus.cc:729] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d72e7db6f44857b2902f8bec7a4810, State: Initialized, Role: FOLLOWER
I20260807 19:38:19.818040 22796 consensus_queue.cc:260] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } }
I20260807 19:38:19.819686 22796 ts_tablet_manager.cc:1428] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810: Time spent starting tablet: real 0.013s user 0.005s sys 0.008s
W20260807 19:38:19.926558 22480 tablet.cc:2369] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:38:19.963366 22625 tablet.cc:2369] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:38:19.988013 22770 tablet.cc:2369] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:38:19.992105 22803 raft_consensus.cc:482] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:38:19.992368 22803 raft_consensus.cc:504] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } }
I20260807 19:38:19.994089 22803 leader_election.cc:288] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783), 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66:45805)
I20260807 19:38:19.996758 22801 raft_consensus.cc:482] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:38:19.997066 22801 raft_consensus.cc:504] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } }
I20260807 19:38:19.998792 22801 leader_election.cc:288] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685), d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783)
I20260807 19:38:20.003096 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.994671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:46235 (local address 127.14.16.66:45805)
0807 19:38:19.994979 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:19.994986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:19.995527 (+ 541us) server_negotiation.cc:407] Connection header received
0807 19:38:19.995771 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:19.995775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:19.995832 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:19.995926 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:19.996528 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.997713 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.000121 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.001394 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.001501 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.001572 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.002639 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.002804 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260807 19:38:20.003624 22792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.994489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:52927 (local address 127.14.16.67:41783)
0807 19:38:19.994736 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:19.994743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:19.995618 (+ 875us) server_negotiation.cc:407] Connection header received
0807 19:38:19.995804 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:19.995808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:19.995864 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:19.995959 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:19.996631 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.997317 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.000103 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.001371 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.001501 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.001572 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.003193 (+ 1621us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.003376 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:38:20.004374 22805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.994365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:45805 (local address 127.14.16.65:46235)
0807 19:38:19.995403 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:19.995422 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:19.995615 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:19.995945 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:19.995950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:19.995982 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:19.996392 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:19.996402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.997858 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.997865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:19.999952 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:19.999965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.001780 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.001788 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.001968 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.002005 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:38:20.003622 (+ 1617us) client_negotiation.cc:240] Negotiation successful
0807 19:38:20.003945 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":854,"threads_started":1}
I20260807 19:38:20.004505 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.994465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:41783 (local address 127.14.16.65:52927)
0807 19:38:19.995521 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:19.995535 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:19.995646 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:19.995975 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:19.995979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:19.996004 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:19.996426 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:19.996442 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:19.997463 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:19.997466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:19.999911 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:19.999924 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.002221 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.002229 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.002436 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.002474 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:38:20.003809 (+ 1335us) client_negotiation.cc:240] Negotiation successful
0807 19:38:20.003967 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":7,"thread_start_us":448,"threads_started":1}
I20260807 19:38:20.005509 22576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8e58de56894eef900d656fc69feea3" candidate_uuid: "64d72e7db6f44857b2902f8bec7a4810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" is_pre_election: true
I20260807 19:38:20.006078 22576 raft_consensus.cc:2454] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64d72e7db6f44857b2902f8bec7a4810 in term 0.
I20260807 19:38:20.006080 22718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8e58de56894eef900d656fc69feea3" candidate_uuid: "64d72e7db6f44857b2902f8bec7a4810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" is_pre_election: true
I20260807 19:38:20.006620 22718 raft_consensus.cc:2454] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64d72e7db6f44857b2902f8bec7a4810 in term 0.
I20260807 19:38:20.007004 22360 leader_election.cc:302] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280e6bd8c951495ba5f75e6f9afe3b97, 64d72e7db6f44857b2902f8bec7a4810; no voters:
I20260807 19:38:20.007449 22803 raft_consensus.cc:2781] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:38:20.007611 22803 raft_consensus.cc:482] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:38:20.007706 22803 raft_consensus.cc:3037] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:20.010224 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.999695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:58891 (local address 127.14.16.67:41783)
0807 19:38:20.002192 (+ 2497us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:20.002200 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:20.003326 (+ 1126us) server_negotiation.cc:407] Connection header received
0807 19:38:20.003499 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:20.003504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:20.003571 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:20.003671 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:20.004635 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.005839 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.008429 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.009233 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.009431 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.009498 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.009772 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.009900 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2333,"thread_start_us":2086,"threads_started":1}
I20260807 19:38:20.010301 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.999262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:41783 (local address 127.14.16.66:58891)
0807 19:38:20.003154 (+ 3892us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:20.003172 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:20.003361 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:20.003967 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:20.003971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:20.003995 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:20.004370 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.004382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.006058 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.006066 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.008219 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.008235 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.009449 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.009455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.009617 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.009640 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:38:20.009747 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 19:38:20.009875 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3745,"mutex_wait_us":2,"thread_start_us":3090,"threads_started":1}
I20260807 19:38:20.010824 22791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.999585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:49713 (local address 127.14.16.65:37685)
0807 19:38:19.999754 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:19.999767 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:20.002825 (+ 3058us) server_negotiation.cc:407] Connection header received
0807 19:38:20.003101 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:20.003108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:20.003214 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:20.003274 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:20.004139 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.005238 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.008388 (+ 3150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.009693 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.009871 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.009937 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.010482 (+ 545us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.010608 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 19:38:20.010979 22718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8e58de56894eef900d656fc69feea3" candidate_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" is_pre_election: true
I20260807 19:38:20.011205 22718 raft_consensus.cc:2454] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 280e6bd8c951495ba5f75e6f9afe3b97 in term 0.
I20260807 19:38:20.011113 22803 raft_consensus.cc:504] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } }
I20260807 19:38:20.011576 22808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:19.999113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:37685 (local address 127.14.16.66:49713)
0807 19:38:20.002721 (+ 3608us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:20.002738 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:20.002945 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:20.003478 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:20.003482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:20.003510 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:20.003954 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.003968 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.005441 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.005447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.008220 (+ 2773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.008236 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.010111 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.010118 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.010299 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.010332 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:38:20.011041 (+ 709us) client_negotiation.cc:240] Negotiation successful
0807 19:38:20.011184 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3444,"mutex_wait_us":82,"thread_start_us":3715,"threads_started":1}
I20260807 19:38:20.011682 22501 leader_election.cc:302] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280e6bd8c951495ba5f75e6f9afe3b97, d2af1c6e12ea424db07c1456c5dcdd5b; no voters:
I20260807 19:38:20.011876 22803 leader_election.cc:288] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [CANDIDATE]: Term 1 election: Requested vote from peers d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783), 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66:45805)
I20260807 19:38:20.012302 22801 raft_consensus.cc:2781] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:38:20.012362 22576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8e58de56894eef900d656fc69feea3" candidate_uuid: "64d72e7db6f44857b2902f8bec7a4810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280e6bd8c951495ba5f75e6f9afe3b97"
I20260807 19:38:20.012472 22801 raft_consensus.cc:482] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:38:20.012560 22801 raft_consensus.cc:3037] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:20.012504 22718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8e58de56894eef900d656fc69feea3" candidate_uuid: "64d72e7db6f44857b2902f8bec7a4810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b"
I20260807 19:38:20.012728 22718 raft_consensus.cc:3037] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:20.012583 22434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8e58de56894eef900d656fc69feea3" candidate_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d72e7db6f44857b2902f8bec7a4810" is_pre_election: true
I20260807 19:38:20.013054 22434 raft_consensus.cc:2379] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 280e6bd8c951495ba5f75e6f9afe3b97 in current term 1: Already voted for candidate 64d72e7db6f44857b2902f8bec7a4810 in this term.
I20260807 19:38:20.014906 22801 raft_consensus.cc:504] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } }
I20260807 19:38:20.015331 22576 raft_consensus.cc:2379] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 64d72e7db6f44857b2902f8bec7a4810 in current term 1: Already voted for candidate 280e6bd8c951495ba5f75e6f9afe3b97 in this term.
I20260807 19:38:20.015611 22718 raft_consensus.cc:2454] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64d72e7db6f44857b2902f8bec7a4810 in term 1.
I20260807 19:38:20.015650 22801 leader_election.cc:288] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [CANDIDATE]: Term 1 election: Requested vote from peers 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685), d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783)
I20260807 19:38:20.016130 22357 leader_election.cc:302] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d72e7db6f44857b2902f8bec7a4810, d2af1c6e12ea424db07c1456c5dcdd5b; no voters:
I20260807 19:38:20.016101 22434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8e58de56894eef900d656fc69feea3" candidate_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d72e7db6f44857b2902f8bec7a4810"
I20260807 19:38:20.016227 22718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f8e58de56894eef900d656fc69feea3" candidate_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b"
I20260807 19:38:20.016433 22434 raft_consensus.cc:2379] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 280e6bd8c951495ba5f75e6f9afe3b97 in current term 1: Already voted for candidate 64d72e7db6f44857b2902f8bec7a4810 in this term.
I20260807 19:38:20.016494 22718 raft_consensus.cc:2379] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 280e6bd8c951495ba5f75e6f9afe3b97 in current term 1: Already voted for candidate 64d72e7db6f44857b2902f8bec7a4810 in this term.
I20260807 19:38:20.016587 22803 raft_consensus.cc:2781] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:20.017189 22501 leader_election.cc:302] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 280e6bd8c951495ba5f75e6f9afe3b97; no voters: 64d72e7db6f44857b2902f8bec7a4810, d2af1c6e12ea424db07c1456c5dcdd5b
I20260807 19:38:20.017313 22803 raft_consensus.cc:686] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 LEADER]: Becoming Leader. State: Replica: 64d72e7db6f44857b2902f8bec7a4810, State: Running, Role: LEADER
I20260807 19:38:20.017596 22801 raft_consensus.cc:2726] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 19:38:20.018148 22803 consensus_queue.cc:237] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } }
I20260807 19:38:20.022396 22264 catalog_manager.cc:5495] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65). New cstate: current_term: 1 leader_uuid: "64d72e7db6f44857b2902f8bec7a4810" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } health_report { overall_health: UNKNOWN } } }
I20260807 19:38:20.039625 22791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.034336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45422 (local address 127.14.16.65:37685)
0807 19:38:20.034554 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:20.034561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:20.034581 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:38:20.034798 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:20.034802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:20.034896 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:20.035048 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:38:20.035457 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.036169 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.037493 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.037898 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.038030 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.038476 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:38:20.038998 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:38:20.039158 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.039218 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.039337 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:38:20.047734 22576 raft_consensus.cc:1261] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Refusing update from remote peer 64d72e7db6f44857b2902f8bec7a4810: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:20.047748 22718 raft_consensus.cc:1261] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Refusing update from remote peer 64d72e7db6f44857b2902f8bec7a4810: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:20.048595 22810 consensus_queue.cc:1035] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:20.048992 22803 consensus_queue.cc:1035] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:20.075089 22265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:20.075438 22265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:20.079275 22265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723: Sending DeleteTablet for 3 replicas of tablet 9f8e58de56894eef900d656fc69feea3
I20260807 19:38:20.080554 22697 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f8e58de56894eef900d656fc69feea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:20 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:52226
I20260807 19:38:20.080551 22407 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f8e58de56894eef900d656fc69feea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:20 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:45414
I20260807 19:38:20.081528 22824 tablet_replica.cc:330] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b: stopping tablet replica
I20260807 19:38:20.081692 22552 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f8e58de56894eef900d656fc69feea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:20 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:44434
I20260807 19:38:20.082149 22825 tablet_replica.cc:330] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810: stopping tablet replica
I20260807 19:38:20.082729 22824 raft_consensus.cc:2229] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:20.083099 22826 tablet_replica.cc:330] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97: stopping tablet replica
I20260807 19:38:20.083173 22825 raft_consensus.cc:2229] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:38:20.083053 22265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:38:20.083650 22826 raft_consensus.cc:2229] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 19:38:20.083791 22265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:38:20.084208 22826 raft_consensus.cc:2258] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:20.084343 22825 raft_consensus.cc:2258] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:20.084929 22824 raft_consensus.cc:2258] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:20.086999 22824 ts_tablet_manager.cc:1905] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:20.086998 22826 ts_tablet_manager.cc:1905] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:20.092046 22825 ts_tablet_manager.cc:1905] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:20.092163 22552 tablet_service.cc:1467] Processing CreateTablet for tablet 11cc3f4955bb4c2093c7f1e9b2cd7338 (DEFAULT_TABLE table=test-table [id=fe8484b1139e4a789fbf515931192648]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:20.092633 22552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cc3f4955bb4c2093c7f1e9b2cd7338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:20.094447 22826 ts_tablet_manager.cc:1918] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:20.094515 22824 ts_tablet_manager.cc:1918] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:20.094626 22826 log.cc:1196] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/9f8e58de56894eef900d656fc69feea3
I20260807 19:38:20.094656 22824 log.cc:1196] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/9f8e58de56894eef900d656fc69feea3
I20260807 19:38:20.095265 22826 ts_tablet_manager.cc:1939] T 9f8e58de56894eef900d656fc69feea3 P 280e6bd8c951495ba5f75e6f9afe3b97: Deleting consensus metadata
I20260807 19:38:20.095268 22824 ts_tablet_manager.cc:1939] T 9f8e58de56894eef900d656fc69feea3 P d2af1c6e12ea424db07c1456c5dcdd5b: Deleting consensus metadata
I20260807 19:38:20.095634 22697 tablet_service.cc:1467] Processing CreateTablet for tablet 11cc3f4955bb4c2093c7f1e9b2cd7338 (DEFAULT_TABLE table=test-table [id=fe8484b1139e4a789fbf515931192648]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:20.096077 22697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cc3f4955bb4c2093c7f1e9b2cd7338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:20.097199 22252 catalog_manager.cc:4841] TS 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66:45805): tablet 9f8e58de56894eef900d656fc69feea3 (table test-table [id=7f2f5b318e434d54849950e4d24da3dd]) successfully deleted
I20260807 19:38:20.098784 22407 tablet_service.cc:1467] Processing CreateTablet for tablet 11cc3f4955bb4c2093c7f1e9b2cd7338 (DEFAULT_TABLE table=test-table [id=fe8484b1139e4a789fbf515931192648]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:20.099241 22407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cc3f4955bb4c2093c7f1e9b2cd7338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:20.102277 22249 catalog_manager.cc:4841] TS d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783): tablet 9f8e58de56894eef900d656fc69feea3 (table test-table [id=7f2f5b318e434d54849950e4d24da3dd]) successfully deleted
I20260807 19:38:20.102670 22795 tablet_bootstrap.cc:492] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b: Bootstrap starting.
I20260807 19:38:20.103976 22797 tablet_bootstrap.cc:492] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97: Bootstrap starting.
I20260807 19:38:20.104322 22795 tablet_bootstrap.cc:654] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:20.105722 22797 tablet_bootstrap.cc:654] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:20.107266 22825 ts_tablet_manager.cc:1918] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:20.107499 22825 log.cc:1196] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/9f8e58de56894eef900d656fc69feea3
I20260807 19:38:20.108111 22825 ts_tablet_manager.cc:1939] T 9f8e58de56894eef900d656fc69feea3 P 64d72e7db6f44857b2902f8bec7a4810: Deleting consensus metadata
I20260807 19:38:20.108980 22796 tablet_bootstrap.cc:492] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810: Bootstrap starting.
I20260807 19:38:20.110128 22251 catalog_manager.cc:4841] TS 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685): tablet 9f8e58de56894eef900d656fc69feea3 (table test-table [id=7f2f5b318e434d54849950e4d24da3dd]) successfully deleted
I20260807 19:38:20.110602 22797 tablet_bootstrap.cc:492] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97: No bootstrap required, opened a new log
I20260807 19:38:20.110692 22796 tablet_bootstrap.cc:654] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:20.110761 22797 ts_tablet_manager.cc:1397] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260807 19:38:20.113093 22797 raft_consensus.cc:348] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.113392 22797 raft_consensus.cc:374] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:20.113481 22797 raft_consensus.cc:729] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280e6bd8c951495ba5f75e6f9afe3b97, State: Initialized, Role: FOLLOWER
I20260807 19:38:20.113878 22797 consensus_queue.cc:260] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.118300 22795 tablet_bootstrap.cc:492] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b: No bootstrap required, opened a new log
I20260807 19:38:20.118417 22797 ts_tablet_manager.cc:1428] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260807 19:38:20.118558 22795 ts_tablet_manager.cc:1397] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.007s
I20260807 19:38:20.119534 22795 raft_consensus.cc:348] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.119843 22795 raft_consensus.cc:374] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:20.120000 22795 raft_consensus.cc:729] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2af1c6e12ea424db07c1456c5dcdd5b, State: Initialized, Role: FOLLOWER
I20260807 19:38:20.120355 22795 consensus_queue.cc:260] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.121023 22795 ts_tablet_manager.cc:1428] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:38:20.122787 22796 tablet_bootstrap.cc:492] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810: No bootstrap required, opened a new log
I20260807 19:38:20.123005 22796 ts_tablet_manager.cc:1397] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.011s
I20260807 19:38:20.123965 22796 raft_consensus.cc:348] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.124217 22796 raft_consensus.cc:374] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:20.124354 22796 raft_consensus.cc:729] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d72e7db6f44857b2902f8bec7a4810, State: Initialized, Role: FOLLOWER
I20260807 19:38:20.124622 22796 consensus_queue.cc:260] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.125182 22796 ts_tablet_manager.cc:1428] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:38:20.214344 22801 raft_consensus.cc:482] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:38:20.214593 22801 raft_consensus.cc:504] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.215242 22801 leader_election.cc:288] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685), d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783)
I20260807 19:38:20.215900 22434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cc3f4955bb4c2093c7f1e9b2cd7338" candidate_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d72e7db6f44857b2902f8bec7a4810" is_pre_election: true
I20260807 19:38:20.215916 22718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cc3f4955bb4c2093c7f1e9b2cd7338" candidate_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" is_pre_election: true
I20260807 19:38:20.216192 22718 raft_consensus.cc:2454] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 280e6bd8c951495ba5f75e6f9afe3b97 in term 0.
I20260807 19:38:20.216202 22434 raft_consensus.cc:2454] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 280e6bd8c951495ba5f75e6f9afe3b97 in term 0.
I20260807 19:38:20.216625 22501 leader_election.cc:302] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280e6bd8c951495ba5f75e6f9afe3b97, d2af1c6e12ea424db07c1456c5dcdd5b; no voters:
I20260807 19:38:20.216945 22801 raft_consensus.cc:2781] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:38:20.217053 22801 raft_consensus.cc:482] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:38:20.217206 22801 raft_consensus.cc:3037] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:20.219470 22801 raft_consensus.cc:504] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.220041 22801 leader_election.cc:288] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [CANDIDATE]: Term 1 election: Requested vote from peers 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685), d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783)
I20260807 19:38:20.220500 22434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cc3f4955bb4c2093c7f1e9b2cd7338" candidate_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d72e7db6f44857b2902f8bec7a4810"
I20260807 19:38:20.220568 22718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cc3f4955bb4c2093c7f1e9b2cd7338" candidate_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b"
I20260807 19:38:20.220746 22434 raft_consensus.cc:3037] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:20.220801 22718 raft_consensus.cc:3037] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:20.223217 22434 raft_consensus.cc:2454] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 280e6bd8c951495ba5f75e6f9afe3b97 in term 1.
I20260807 19:38:20.223744 22503 leader_election.cc:302] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280e6bd8c951495ba5f75e6f9afe3b97, 64d72e7db6f44857b2902f8bec7a4810; no voters:
I20260807 19:38:20.223877 22718 raft_consensus.cc:2454] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 280e6bd8c951495ba5f75e6f9afe3b97 in term 1.
I20260807 19:38:20.224040 22801 raft_consensus.cc:2781] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:20.227252 22801 raft_consensus.cc:686] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 LEADER]: Becoming Leader. State: Replica: 280e6bd8c951495ba5f75e6f9afe3b97, State: Running, Role: LEADER
I20260807 19:38:20.228010 22801 consensus_queue.cc:237] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.231745 22264 catalog_manager.cc:5495] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 reported cstate change: term changed from 0 to 1, leader changed from <none> to 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66). New cstate: current_term: 1 leader_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } health_report { overall_health: UNKNOWN } } }
Aug 07 19:38:20 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131496, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.126@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:20.297297 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.281617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39419 (local address 127.0.0.1:35706)
0807 19:38:20.282671 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:20.282688 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:20.282858 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:20.283351 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:20.283356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:20.284014 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:20.284202 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.284212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.285811 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.285819 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.287425 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.287443 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.288211 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.288217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.288387 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.289117 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:20.289135 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:20.291414 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:20.294574 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:20.294589 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:20.294597 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:20.295039 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:20.295474 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:20.295480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:20.295484 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:20.295635 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:20.296173 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:20.296180 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:20.296354 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 19:38:20.296515 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 19:38:20.296837 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"thread_start_us":774,"threads_started":1}
I20260807 19:38:20.297345 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.281693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35706 (local address 127.14.16.126:39419)
0807 19:38:20.282386 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:20.282392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:20.282778 (+ 386us) server_negotiation.cc:407] Connection header received
0807 19:38:20.283002 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:20.283009 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:20.283089 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:20.283217 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:20.284354 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.285603 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.287599 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.288032 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.288187 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.291605 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:20.291610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:20.291613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:20.291642 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:20.294452 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:20.295170 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:20.295174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:20.295180 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:20.295369 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:20.295752 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:20.295756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:20.295760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:20.296045 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:20.296200 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.296607 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.296961 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":165,"server-negotiator.queue_time_us":512,"thread_start_us":327,"threads_started":1}
I20260807 19:38:20.302438 22264 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:35706:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 19:38:20.303499 22264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:38:20.310554 22407 tablet_service.cc:1467] Processing CreateTablet for tablet df764373d3ca4c4ab169b4e281f3a513 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e3660eeb4cc4d11b2f48d68a803ed69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:38:20.310708 22552 tablet_service.cc:1467] Processing CreateTablet for tablet df764373d3ca4c4ab169b4e281f3a513 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e3660eeb4cc4d11b2f48d68a803ed69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:38:20.311169 22407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df764373d3ca4c4ab169b4e281f3a513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:20.311352 22552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df764373d3ca4c4ab169b4e281f3a513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:20.316638 22797 tablet_bootstrap.cc:492] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: Bootstrap starting.
I20260807 19:38:20.316475 22697 tablet_service.cc:1467] Processing CreateTablet for tablet df764373d3ca4c4ab169b4e281f3a513 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e3660eeb4cc4d11b2f48d68a803ed69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:38:20.317022 22697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df764373d3ca4c4ab169b4e281f3a513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:20.318483 22797 tablet_bootstrap.cc:654] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:20.324309 22796 tablet_bootstrap.cc:492] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810: Bootstrap starting.
I20260807 19:38:20.324860 22795 tablet_bootstrap.cc:492] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b: Bootstrap starting.
I20260807 19:38:20.326287 22796 tablet_bootstrap.cc:654] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:20.326543 22795 tablet_bootstrap.cc:654] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:20.326948 22797 tablet_bootstrap.cc:492] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: No bootstrap required, opened a new log
I20260807 19:38:20.327173 22797 ts_tablet_manager.cc:1397] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260807 19:38:20.328241 22797 raft_consensus.cc:348] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.328673 22797 raft_consensus.cc:374] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:20.328824 22797 raft_consensus.cc:729] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280e6bd8c951495ba5f75e6f9afe3b97, State: Initialized, Role: FOLLOWER
I20260807 19:38:20.329290 22797 consensus_queue.cc:260] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.331182 22801 tablet_replica.cc:439] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } }
I20260807 19:38:20.331583 22801 tablet_replica.cc:442] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:38:20.332479 22797 ts_tablet_manager.cc:1428] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 19:38:20.335340 22795 tablet_bootstrap.cc:492] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b: No bootstrap required, opened a new log
I20260807 19:38:20.335614 22795 ts_tablet_manager.cc:1397] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 19:38:20.336617 22795 raft_consensus.cc:348] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.336932 22795 raft_consensus.cc:374] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:20.337031 22795 raft_consensus.cc:729] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2af1c6e12ea424db07c1456c5dcdd5b, State: Initialized, Role: FOLLOWER
I20260807 19:38:20.337361 22795 consensus_queue.cc:260] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.338001 22795 ts_tablet_manager.cc:1428] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 19:38:20.337844 22802 tablet_replica.cc:439] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } }
I20260807 19:38:20.338245 22802 tablet_replica.cc:442] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:38:20.338521 22796 tablet_bootstrap.cc:492] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810: No bootstrap required, opened a new log
I20260807 19:38:20.338721 22796 ts_tablet_manager.cc:1397] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.008s
I20260807 19:38:20.339569 22796 raft_consensus.cc:348] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.339831 22796 raft_consensus.cc:374] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:20.339921 22796 raft_consensus.cc:729] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d72e7db6f44857b2902f8bec7a4810, State: Initialized, Role: FOLLOWER
I20260807 19:38:20.340198 22796 consensus_queue.cc:260] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.341131 22818 tablet_replica.cc:439] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } }
I20260807 19:38:20.343047 22796 ts_tablet_manager.cc:1428] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 19:38:20.343111 22818 tablet_replica.cc:442] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:38:20.368711 22801 raft_consensus.cc:482] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:38:20.368918 22801 raft_consensus.cc:504] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.369609 22801 leader_election.cc:288] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685), d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783)
I20260807 19:38:20.370061 22434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df764373d3ca4c4ab169b4e281f3a513" candidate_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d72e7db6f44857b2902f8bec7a4810" is_pre_election: true
I20260807 19:38:20.370229 22718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df764373d3ca4c4ab169b4e281f3a513" candidate_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" is_pre_election: true
I20260807 19:38:20.370395 22434 raft_consensus.cc:2454] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 280e6bd8c951495ba5f75e6f9afe3b97 in term 0.
I20260807 19:38:20.370515 22718 raft_consensus.cc:2454] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 280e6bd8c951495ba5f75e6f9afe3b97 in term 0.
I20260807 19:38:20.370836 22503 leader_election.cc:302] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280e6bd8c951495ba5f75e6f9afe3b97, 64d72e7db6f44857b2902f8bec7a4810; no voters:
I20260807 19:38:20.371093 22801 raft_consensus.cc:2781] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:38:20.371243 22801 raft_consensus.cc:482] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:38:20.371397 22801 raft_consensus.cc:3037] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:20.373440 22801 raft_consensus.cc:504] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.374064 22801 leader_election.cc:288] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [CANDIDATE]: Term 1 election: Requested vote from peers 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685), d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783)
I20260807 19:38:20.374477 22434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df764373d3ca4c4ab169b4e281f3a513" candidate_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d72e7db6f44857b2902f8bec7a4810"
I20260807 19:38:20.374571 22718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df764373d3ca4c4ab169b4e281f3a513" candidate_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b"
I20260807 19:38:20.374758 22434 raft_consensus.cc:3037] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:20.374892 22718 raft_consensus.cc:3037] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:20.377074 22718 raft_consensus.cc:2454] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 280e6bd8c951495ba5f75e6f9afe3b97 in term 1.
I20260807 19:38:20.377511 22501 leader_election.cc:302] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280e6bd8c951495ba5f75e6f9afe3b97, d2af1c6e12ea424db07c1456c5dcdd5b; no voters:
I20260807 19:38:20.377821 22801 raft_consensus.cc:2781] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:20.377993 22801 raft_consensus.cc:686] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 LEADER]: Becoming Leader. State: Replica: 280e6bd8c951495ba5f75e6f9afe3b97, State: Running, Role: LEADER
I20260807 19:38:20.378011 22434 raft_consensus.cc:2454] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 280e6bd8c951495ba5f75e6f9afe3b97 in term 1.
I20260807 19:38:20.378259 22801 consensus_queue.cc:237] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.379931 22830 tablet_replica.cc:439] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: TxnStatusTablet state changed. Reason: New leader 280e6bd8c951495ba5f75e6f9afe3b97. Latest consensus state: current_term: 1 leader_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } }
I20260807 19:38:20.380362 22830 tablet_replica.cc:442] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:38:20.381047 22843 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 19:38:20.381307 22843 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 19:38:20.382671 22264 catalog_manager.cc:5495] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 reported cstate change: term changed from 0 to 1, leader changed from <none> to 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66). New cstate: current_term: 1 leader_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } health_report { overall_health: UNKNOWN } } }
I20260807 19:38:20.418386 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.410895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44438 (local address 127.14.16.66:45805)
0807 19:38:20.411086 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:20.411092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:20.411116 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:38:20.411423 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:20.411429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:20.411558 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:20.411734 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:38:20.412176 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.413453 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.415448 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.416132 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.416272 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.416820 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:38:20.417548 (+ 728us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:38:20.417704 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.417918 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.418015 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:38:20.418453 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.410771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:45805 (local address 127.0.0.1:44438)
0807 19:38:20.411000 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:20.411021 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:20.411157 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:20.411790 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:20.411795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:20.411809 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:38:20.412010 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.412021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.413601 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.413607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.415232 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.415248 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.416308 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.416315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.416529 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.416602 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:38:20.417744 (+ 1142us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:38:20.417769 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:38:20.417866 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 19:38:20.418132 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 19:38:20.423436 22434 raft_consensus.cc:1261] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Refusing update from remote peer 280e6bd8c951495ba5f75e6f9afe3b97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:20.424408 22830 consensus_queue.cc:1035] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:20.424930 22718 raft_consensus.cc:1261] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Refusing update from remote peer 280e6bd8c951495ba5f75e6f9afe3b97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:20.427793 22830 consensus_queue.cc:1035] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:20.437494 22818 tablet_replica.cc:439] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810: TxnStatusTablet state changed. Reason: New leader 280e6bd8c951495ba5f75e6f9afe3b97. Latest consensus state: current_term: 1 leader_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } }
I20260807 19:38:20.437831 22818 tablet_replica.cc:442] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:38:20.439378 22801 tablet_replica.cc:439] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } }
I20260807 19:38:20.439708 22801 tablet_replica.cc:442] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:38:20.440999 22813 mvcc.cc:205] Tried to move back new op lower bound from 7315994625724850176 to 7315994625551564800. Current Snapshot: MvccSnapshot[applied={T|T < 7315994625724850176}]
I20260807 19:38:20.440958 22815 mvcc.cc:205] Tried to move back new op lower bound from 7315994625724850176 to 7315994625551564800. Current Snapshot: MvccSnapshot[applied={T|T < 7315994625724850176 or (T in {7315994625724850176})}]
I20260807 19:38:20.442198 22818 tablet_replica.cc:439] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } }
I20260807 19:38:20.442431 22818 tablet_replica.cc:442] T df764373d3ca4c4ab169b4e281f3a513 P 64d72e7db6f44857b2902f8bec7a4810: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:38:20.448180 22802 tablet_replica.cc:439] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b: TxnStatusTablet state changed. Reason: New leader 280e6bd8c951495ba5f75e6f9afe3b97. Latest consensus state: current_term: 1 leader_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } }
I20260807 19:38:20.448491 22802 tablet_replica.cc:442] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:38:20.457105 22830 tablet_replica.cc:439] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } }
I20260807 19:38:20.457397 22830 tablet_replica.cc:442] T df764373d3ca4c4ab169b4e281f3a513 P 280e6bd8c951495ba5f75e6f9afe3b97: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:38:20.460027 22814 mvcc.cc:205] Tried to move back new op lower bound from 7315994625724850176 to 7315994625551564800. Current Snapshot: MvccSnapshot[applied={T|T < 7315994625724850176 or (T in {7315994625724850176})}]
I20260807 19:38:20.460852 22802 tablet_replica.cc:439] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } } }
I20260807 19:38:20.461112 22802 tablet_replica.cc:442] T df764373d3ca4c4ab169b4e281f3a513 P d2af1c6e12ea424db07c1456c5dcdd5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:38:20.468118 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.451905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44448 (local address 127.14.16.66:45805)
0807 19:38:20.452079 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:20.452085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:20.452102 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:38:20.452181 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:20.452186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:20.452278 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:20.452367 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:38:20.452951 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.454150 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.462788 (+ 8638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.463552 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.463674 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.466882 (+ 3208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:38:20.467011 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:38:20.467091 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.467694 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.467799 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:38:20.489652 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.481960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:34741 (local address 127.14.16.66:45805)
0807 19:38:20.482159 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:20.482165 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:20.482460 (+ 295us) server_negotiation.cc:407] Connection header received
0807 19:38:20.482794 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:20.482800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:20.482885 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:20.482977 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:38:20.483836 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.484991 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.487208 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.487989 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.488109 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.488787 (+ 678us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:38:20.488942 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:38:20.489011 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.489205 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.489302 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":52}
I20260807 19:38:20.490423 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.481679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:45805 (local address 127.14.16.66:34741)
0807 19:38:20.482368 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:20.482388 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:20.482582 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:20.483027 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:20.483032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:20.483045 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:38:20.483636 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.483651 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.485173 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.485178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.487055 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.487070 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.488313 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.488322 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.488487 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.488595 (+ 108us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:38:20.489084 (+ 489us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:38:20.489092 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:38:20.489852 (+ 760us) client_negotiation.cc:240] Negotiation successful
0807 19:38:20.489996 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":24,"thread_start_us":488,"threads_started":1}
I20260807 19:38:20.503860 22718 raft_consensus.cc:1261] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Refusing update from remote peer 280e6bd8c951495ba5f75e6f9afe3b97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:20.504658 22830 consensus_queue.cc:1035] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:20.509183 22434 raft_consensus.cc:1261] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Refusing update from remote peer 280e6bd8c951495ba5f75e6f9afe3b97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:20.511349 22801 consensus_queue.cc:1035] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:20.620265 22264 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:20.620561 22264 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:20.623910 22264 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723: Sending DeleteTablet for 3 replicas of tablet 11cc3f4955bb4c2093c7f1e9b2cd7338
I20260807 19:38:20.624795 22407 tablet_service.cc:1514] Processing DeleteTablet for tablet 11cc3f4955bb4c2093c7f1e9b2cd7338 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:20 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:45414
I20260807 19:38:20.624943 22551 tablet_service.cc:1514] Processing DeleteTablet for tablet 11cc3f4955bb4c2093c7f1e9b2cd7338 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:20 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:44434
I20260807 19:38:20.625242 22697 tablet_service.cc:1514] Processing DeleteTablet for tablet 11cc3f4955bb4c2093c7f1e9b2cd7338 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:20 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:52226
I20260807 19:38:20.625501 22856 tablet_replica.cc:330] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810: stopping tablet replica
I20260807 19:38:20.625710 22856 raft_consensus.cc:2229] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:20.625937 22856 raft_consensus.cc:2258] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:20.627115 22856 ts_tablet_manager.cc:1905] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:20.634786 22856 ts_tablet_manager.cc:1918] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:38:20.634960 22856 log.cc:1196] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/11cc3f4955bb4c2093c7f1e9b2cd7338
I20260807 19:38:20.635496 22856 ts_tablet_manager.cc:1939] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 64d72e7db6f44857b2902f8bec7a4810: Deleting consensus metadata
I20260807 19:38:20.636827 22251 catalog_manager.cc:4841] TS 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685): tablet 11cc3f4955bb4c2093c7f1e9b2cd7338 (table test-table [id=fe8484b1139e4a789fbf515931192648]) successfully deleted
I20260807 19:38:20.645772 22858 tablet_replica.cc:330] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b: stopping tablet replica
I20260807 19:38:20.646014 22858 raft_consensus.cc:2229] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:20.646288 22858 raft_consensus.cc:2258] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:20.646878 22857 tablet_replica.cc:330] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97: stopping tablet replica
I20260807 19:38:20.647131 22857 raft_consensus.cc:2229] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:38:20.647663 22858 ts_tablet_manager.cc:1905] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:20.647795 22857 raft_consensus.cc:2258] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:20.648804 22857 ts_tablet_manager.cc:1905] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:20.654050 22858 ts_tablet_manager.cc:1918] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:38:20.654196 22858 log.cc:1196] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/11cc3f4955bb4c2093c7f1e9b2cd7338
I20260807 19:38:20.654660 22858 ts_tablet_manager.cc:1939] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P d2af1c6e12ea424db07c1456c5dcdd5b: Deleting consensus metadata
I20260807 19:38:20.655922 22249 catalog_manager.cc:4841] TS d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783): tablet 11cc3f4955bb4c2093c7f1e9b2cd7338 (table test-table [id=fe8484b1139e4a789fbf515931192648]) successfully deleted
I20260807 19:38:20.656384 22857 ts_tablet_manager.cc:1918] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:38:20.656541 22857 log.cc:1196] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/11cc3f4955bb4c2093c7f1e9b2cd7338
I20260807 19:38:20.657426 22857 ts_tablet_manager.cc:1939] T 11cc3f4955bb4c2093c7f1e9b2cd7338 P 280e6bd8c951495ba5f75e6f9afe3b97: Deleting consensus metadata
I20260807 19:38:20.658638 22252 catalog_manager.cc:4841] TS 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66:45805): tablet 11cc3f4955bb4c2093c7f1e9b2cd7338 (table test-table [id=fe8484b1139e4a789fbf515931192648]) successfully deleted
Aug 07 19:38:20 dist-test-slave-07c2 krb5kdc[22220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:38:20 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:20 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:38:20.703501 22838 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:20.692096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35714 (local address 127.14.16.126:39419)
0807 19:38:20.692347 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:20.692354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:20.692646 (+ 292us) server_negotiation.cc:407] Connection header received
0807 19:38:20.692847 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:20.692854 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:20.692968 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:20.693122 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:20.694560 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.695489 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.696750 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.697185 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.697325 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.703208 (+ 5883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35714: BlockingRecv error: recv got EOF from 127.0.0.1:35714 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 19:38:20 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM
I20260807 19:38:20.723658 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.706907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35726 (local address 127.14.16.126:39419)
0807 19:38:20.707129 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:20.707135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:20.707455 (+ 320us) server_negotiation.cc:407] Connection header received
0807 19:38:20.707579 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:20.707585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:20.707668 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:20.707792 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:20.708913 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.710176 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.711421 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.712162 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.712320 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.716227 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:20.716235 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:20.716241 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:20.716304 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:20.720494 (+ 4190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:20.721177 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:20.721183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:20.721187 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:20.721403 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:20.721837 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:20.721843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:20.721846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:20.722234 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:20.722526 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.722906 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.723172 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 19:38:20.727826 22264 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:38:20.728626 22264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:38:20.736202 22407 tablet_service.cc:1467] Processing CreateTablet for tablet 1d2f20177b944fe8b675fb3f80c38590 (DEFAULT_TABLE table=test-table [id=c6ac383ea3364ea3baa697c153c7edb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:20.736523 22697 tablet_service.cc:1467] Processing CreateTablet for tablet 1d2f20177b944fe8b675fb3f80c38590 (DEFAULT_TABLE table=test-table [id=c6ac383ea3364ea3baa697c153c7edb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:20.736696 22407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d2f20177b944fe8b675fb3f80c38590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:20.736938 22697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d2f20177b944fe8b675fb3f80c38590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:20.739986 22551 tablet_service.cc:1467] Processing CreateTablet for tablet 1d2f20177b944fe8b675fb3f80c38590 (DEFAULT_TABLE table=test-table [id=c6ac383ea3364ea3baa697c153c7edb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:20.740628 22551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d2f20177b944fe8b675fb3f80c38590. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:20.743216 22795 tablet_bootstrap.cc:492] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b: Bootstrap starting.
I20260807 19:38:20.743216 22796 tablet_bootstrap.cc:492] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810: Bootstrap starting.
I20260807 19:38:20.744915 22795 tablet_bootstrap.cc:654] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:20.744915 22796 tablet_bootstrap.cc:654] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:20.749491 22797 tablet_bootstrap.cc:492] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97: Bootstrap starting.
I20260807 19:38:20.751521 22797 tablet_bootstrap.cc:654] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:20.754806 22797 tablet_bootstrap.cc:492] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97: No bootstrap required, opened a new log
I20260807 19:38:20.755031 22795 tablet_bootstrap.cc:492] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b: No bootstrap required, opened a new log
I20260807 19:38:20.755182 22797 ts_tablet_manager.cc:1397] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 19:38:20.755195 22795 ts_tablet_manager.cc:1397] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260807 19:38:20.755820 22796 tablet_bootstrap.cc:492] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810: No bootstrap required, opened a new log
I20260807 19:38:20.755966 22796 ts_tablet_manager.cc:1397] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260807 19:38:20.756055 22795 raft_consensus.cc:348] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.756323 22795 raft_consensus.cc:374] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:20.756425 22795 raft_consensus.cc:729] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2af1c6e12ea424db07c1456c5dcdd5b, State: Initialized, Role: FOLLOWER
I20260807 19:38:20.756477 22797 raft_consensus.cc:348] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.756703 22795 consensus_queue.cc:260] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.756798 22796 raft_consensus.cc:348] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.757054 22796 raft_consensus.cc:374] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:20.757145 22796 raft_consensus.cc:729] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d72e7db6f44857b2902f8bec7a4810, State: Initialized, Role: FOLLOWER
I20260807 19:38:20.757376 22795 ts_tablet_manager.cc:1428] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:38:20.757377 22796 consensus_queue.cc:260] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.757920 22796 ts_tablet_manager.cc:1428] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 19:38:20.758358 22797 raft_consensus.cc:374] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:20.758563 22797 raft_consensus.cc:729] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280e6bd8c951495ba5f75e6f9afe3b97, State: Initialized, Role: FOLLOWER
I20260807 19:38:20.758934 22797 consensus_queue.cc:260] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.763320 22797 ts_tablet_manager.cc:1428] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260807 19:38:20.885064 22802 raft_consensus.cc:482] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:38:20.885325 22802 raft_consensus.cc:504] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.886972 22802 leader_election.cc:288] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685), 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66:45805)
I20260807 19:38:20.907500 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.887459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:47227 (local address 127.14.16.66:45805)
0807 19:38:20.887709 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:20.887718 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:20.900002 (+ 12284us) server_negotiation.cc:407] Connection header received
0807 19:38:20.900565 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:20.900575 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:20.900702 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:20.900792 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:20.901718 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.902896 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.905301 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.906481 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.906652 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.906727 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.907070 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.907201 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:38:20.907647 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.887229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:45805 (local address 127.14.16.67:47227)
0807 19:38:20.899869 (+ 12640us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:20.899896 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:20.900343 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:20.901057 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:20.901065 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:20.901100 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:20.901507 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.901522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.903041 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.903047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.905128 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.905151 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.906694 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.906700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.906917 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.906941 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:38:20.907046 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 19:38:20.907178 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12452,"mutex_wait_us":21,"thread_start_us":14843,"threads_started":1}
I20260807 19:38:20.908301 22576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d2f20177b944fe8b675fb3f80c38590" candidate_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" is_pre_election: true
I20260807 19:38:20.908620 22576 raft_consensus.cc:2454] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2af1c6e12ea424db07c1456c5dcdd5b in term 0.
I20260807 19:38:20.909641 22874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.887242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:37685 (local address 127.14.16.67:33391)
0807 19:38:20.902334 (+ 15092us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:20.902353 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:20.902473 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:20.902821 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:20.902826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:20.902852 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:20.903261 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.903272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.904781 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.904787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.907016 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:20.907031 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.908688 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.908694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:20.908981 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.909004 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:38:20.909113 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 19:38:20.909230 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14965,"mutex_wait_us":635,"thread_start_us":12167,"threads_started":1}
I20260807 19:38:20.909636 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.887491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:33391 (local address 127.14.16.65:37685)
0807 19:38:20.902109 (+ 14618us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:20.902116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:20.902453 (+ 337us) server_negotiation.cc:407] Connection header received
0807 19:38:20.902608 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:20.902612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:20.902684 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:20.902784 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:20.903481 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.904597 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.907221 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.908489 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.908667 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.908739 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.909144 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.909293 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14469,"thread_start_us":14371,"threads_started":1}
I20260807 19:38:20.909591 22648 leader_election.cc:302] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280e6bd8c951495ba5f75e6f9afe3b97, d2af1c6e12ea424db07c1456c5dcdd5b; no voters:
I20260807 19:38:20.910293 22802 raft_consensus.cc:2781] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:38:20.910421 22802 raft_consensus.cc:482] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:38:20.910508 22802 raft_consensus.cc:3037] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:20.910583 22433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d2f20177b944fe8b675fb3f80c38590" candidate_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d72e7db6f44857b2902f8bec7a4810" is_pre_election: true
I20260807 19:38:20.910846 22433 raft_consensus.cc:2454] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2af1c6e12ea424db07c1456c5dcdd5b in term 0.
I20260807 19:38:20.912981 22802 raft_consensus.cc:504] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.913602 22802 leader_election.cc:288] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [CANDIDATE]: Term 1 election: Requested vote from peers 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685), 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66:45805)
I20260807 19:38:20.913981 22433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d2f20177b944fe8b675fb3f80c38590" candidate_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d72e7db6f44857b2902f8bec7a4810"
I20260807 19:38:20.914201 22433 raft_consensus.cc:3037] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:20.914124 22576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d2f20177b944fe8b675fb3f80c38590" candidate_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280e6bd8c951495ba5f75e6f9afe3b97"
I20260807 19:38:20.914381 22576 raft_consensus.cc:3037] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:20.917251 22433 raft_consensus.cc:2454] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2af1c6e12ea424db07c1456c5dcdd5b in term 1.
I20260807 19:38:20.917593 22576 raft_consensus.cc:2454] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2af1c6e12ea424db07c1456c5dcdd5b in term 1.
I20260807 19:38:20.917650 22647 leader_election.cc:302] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d72e7db6f44857b2902f8bec7a4810, d2af1c6e12ea424db07c1456c5dcdd5b; no voters:
I20260807 19:38:20.917949 22802 raft_consensus.cc:2781] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:20.918527 22802 raft_consensus.cc:686] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 LEADER]: Becoming Leader. State: Replica: d2af1c6e12ea424db07c1456c5dcdd5b, State: Running, Role: LEADER
I20260807 19:38:20.919401 22802 consensus_queue.cc:237] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:20.923178 22264 catalog_manager.cc:5495] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b reported cstate change: term changed from 0 to 1, leader changed from <none> to d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67). New cstate: current_term: 1 leader_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } health_report { overall_health: HEALTHY } } }
I20260807 19:38:20.984995 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:20.967769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.14.16.67:41783)
0807 19:38:20.979133 (+ 11364us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:20.979138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:20.979159 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:38:20.979246 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:20.979249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:20.979394 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:20.979583 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:38:20.980046 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.980855 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.982453 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:20.982973 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:20.983185 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:20.983752 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:38:20.984323 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:38:20.984478 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:20.984577 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 19:38:20.984658 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11196,"thread_start_us":11051,"threads_started":1}
I20260807 19:38:20.988628 22576 raft_consensus.cc:1261] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Refusing update from remote peer d2af1c6e12ea424db07c1456c5dcdd5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:20.988799 22433 raft_consensus.cc:1261] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Refusing update from remote peer d2af1c6e12ea424db07c1456c5dcdd5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:20.989425 22877 consensus_queue.cc:1035] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:20.990329 22877 consensus_queue.cc:1035] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:21.019420 22265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:21.019685 22265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:21.022991 22265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723: Sending DeleteTablet for 3 replicas of tablet 1d2f20177b944fe8b675fb3f80c38590
I20260807 19:38:21.023799 22407 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d2f20177b944fe8b675fb3f80c38590 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:21 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:45414
I20260807 19:38:21.024029 22697 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d2f20177b944fe8b675fb3f80c38590 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:21 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:52226
I20260807 19:38:21.024335 22856 tablet_replica.cc:330] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810: stopping tablet replica
I20260807 19:38:21.024387 22858 tablet_replica.cc:330] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b: stopping tablet replica
I20260807 19:38:21.024544 22856 raft_consensus.cc:2229] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:21.024596 22858 raft_consensus.cc:2229] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:38:21.024762 22856 raft_consensus.cc:2258] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:21.025048 22858 raft_consensus.cc:2258] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:21.026146 22858 ts_tablet_manager.cc:1905] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:21.026324 22551 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d2f20177b944fe8b675fb3f80c38590 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:21 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:44434
I20260807 19:38:21.026615 22857 tablet_replica.cc:330] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97: stopping tablet replica
I20260807 19:38:21.026787 22857 raft_consensus.cc:2229] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:21.027015 22857 raft_consensus.cc:2258] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:21.027642 22265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:38:21.028079 22856 ts_tablet_manager.cc:1905] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 19:38:21.028316 22265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:38:21.030194 22857 ts_tablet_manager.cc:1905] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:21.033699 22858 ts_tablet_manager.cc:1918] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:21.033981 22858 log.cc:1196] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/1d2f20177b944fe8b675fb3f80c38590
I20260807 19:38:21.034914 22856 ts_tablet_manager.cc:1918] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:21.035063 22856 log.cc:1196] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/1d2f20177b944fe8b675fb3f80c38590
I20260807 19:38:21.035501 22856 ts_tablet_manager.cc:1939] T 1d2f20177b944fe8b675fb3f80c38590 P 64d72e7db6f44857b2902f8bec7a4810: Deleting consensus metadata
I20260807 19:38:21.035800 22858 ts_tablet_manager.cc:1939] T 1d2f20177b944fe8b675fb3f80c38590 P d2af1c6e12ea424db07c1456c5dcdd5b: Deleting consensus metadata
I20260807 19:38:21.036581 22251 catalog_manager.cc:4841] TS 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685): tablet 1d2f20177b944fe8b675fb3f80c38590 (table test-table [id=c6ac383ea3364ea3baa697c153c7edb4]) successfully deleted
I20260807 19:38:21.037389 22249 catalog_manager.cc:4841] TS d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783): tablet 1d2f20177b944fe8b675fb3f80c38590 (table test-table [id=c6ac383ea3364ea3baa697c153c7edb4]) successfully deleted
I20260807 19:38:21.039474 22407 tablet_service.cc:1467] Processing CreateTablet for tablet c504c34517404586ac3ba51a03da9cb1 (DEFAULT_TABLE table=test-table [id=cdd9e449bde34eb89a7bcb287ba77419]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:21.039901 22407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c504c34517404586ac3ba51a03da9cb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:21.039825 22697 tablet_service.cc:1467] Processing CreateTablet for tablet c504c34517404586ac3ba51a03da9cb1 (DEFAULT_TABLE table=test-table [id=cdd9e449bde34eb89a7bcb287ba77419]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:21.040234 22697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c504c34517404586ac3ba51a03da9cb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:21.044212 22551 tablet_service.cc:1467] Processing CreateTablet for tablet c504c34517404586ac3ba51a03da9cb1 (DEFAULT_TABLE table=test-table [id=cdd9e449bde34eb89a7bcb287ba77419]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:21.044682 22551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c504c34517404586ac3ba51a03da9cb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:21.046571 22796 tablet_bootstrap.cc:492] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810: Bootstrap starting.
I20260807 19:38:21.048130 22796 tablet_bootstrap.cc:654] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:21.053943 22797 tablet_bootstrap.cc:492] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97: Bootstrap starting.
I20260807 19:38:21.055732 22797 tablet_bootstrap.cc:654] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:21.056134 22795 tablet_bootstrap.cc:492] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b: Bootstrap starting.
I20260807 19:38:21.058302 22795 tablet_bootstrap.cc:654] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:21.061244 22795 tablet_bootstrap.cc:492] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b: No bootstrap required, opened a new log
I20260807 19:38:21.061416 22795 ts_tablet_manager.cc:1397] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 19:38:21.062129 22795 raft_consensus.cc:348] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:21.062436 22795 raft_consensus.cc:374] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:21.062511 22795 raft_consensus.cc:729] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2af1c6e12ea424db07c1456c5dcdd5b, State: Initialized, Role: FOLLOWER
I20260807 19:38:21.062562 22857 ts_tablet_manager.cc:1918] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:21.062889 22857 log.cc:1196] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/1d2f20177b944fe8b675fb3f80c38590
I20260807 19:38:21.062816 22795 consensus_queue.cc:260] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:21.063558 22857 ts_tablet_manager.cc:1939] T 1d2f20177b944fe8b675fb3f80c38590 P 280e6bd8c951495ba5f75e6f9afe3b97: Deleting consensus metadata
I20260807 19:38:21.064167 22795 ts_tablet_manager.cc:1428] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 19:38:21.065774 22252 catalog_manager.cc:4841] TS 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66:45805): tablet 1d2f20177b944fe8b675fb3f80c38590 (table test-table [id=c6ac383ea3364ea3baa697c153c7edb4]) successfully deleted
I20260807 19:38:21.074194 22797 tablet_bootstrap.cc:492] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97: No bootstrap required, opened a new log
I20260807 19:38:21.074378 22797 ts_tablet_manager.cc:1397] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.009s
I20260807 19:38:21.075204 22797 raft_consensus.cc:348] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:21.075505 22797 raft_consensus.cc:374] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:21.075634 22797 raft_consensus.cc:729] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280e6bd8c951495ba5f75e6f9afe3b97, State: Initialized, Role: FOLLOWER
I20260807 19:38:21.075924 22797 consensus_queue.cc:260] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:21.079550 22797 ts_tablet_manager.cc:1428] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260807 19:38:21.083250 22796 tablet_bootstrap.cc:492] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810: No bootstrap required, opened a new log
I20260807 19:38:21.083479 22796 ts_tablet_manager.cc:1397] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810: Time spent bootstrapping tablet: real 0.037s user 0.009s sys 0.021s
I20260807 19:38:21.084280 22796 raft_consensus.cc:348] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:21.084548 22796 raft_consensus.cc:374] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:21.084626 22796 raft_consensus.cc:729] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d72e7db6f44857b2902f8bec7a4810, State: Initialized, Role: FOLLOWER
I20260807 19:38:21.084877 22796 consensus_queue.cc:260] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:21.085444 22796 ts_tablet_manager.cc:1428] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 19:38:21.185469 22818 raft_consensus.cc:482] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:38:21.185767 22818 raft_consensus.cc:504] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:21.186565 22818 leader_election.cc:288] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66:45805), d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783)
I20260807 19:38:21.187103 22576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c504c34517404586ac3ba51a03da9cb1" candidate_uuid: "64d72e7db6f44857b2902f8bec7a4810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" is_pre_election: true
I20260807 19:38:21.187343 22717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c504c34517404586ac3ba51a03da9cb1" candidate_uuid: "64d72e7db6f44857b2902f8bec7a4810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" is_pre_election: true
I20260807 19:38:21.187547 22576 raft_consensus.cc:2454] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64d72e7db6f44857b2902f8bec7a4810 in term 0.
I20260807 19:38:21.187703 22717 raft_consensus.cc:2454] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64d72e7db6f44857b2902f8bec7a4810 in term 0.
I20260807 19:38:21.188134 22360 leader_election.cc:302] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280e6bd8c951495ba5f75e6f9afe3b97, 64d72e7db6f44857b2902f8bec7a4810; no voters:
I20260807 19:38:21.188452 22818 raft_consensus.cc:2781] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:38:21.188596 22818 raft_consensus.cc:482] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:38:21.188714 22818 raft_consensus.cc:3037] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:21.191905 22818 raft_consensus.cc:504] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:21.193045 22576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c504c34517404586ac3ba51a03da9cb1" candidate_uuid: "64d72e7db6f44857b2902f8bec7a4810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280e6bd8c951495ba5f75e6f9afe3b97"
I20260807 19:38:21.193187 22717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c504c34517404586ac3ba51a03da9cb1" candidate_uuid: "64d72e7db6f44857b2902f8bec7a4810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b"
I20260807 19:38:21.193302 22576 raft_consensus.cc:3037] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:21.193425 22717 raft_consensus.cc:3037] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:21.196590 22576 raft_consensus.cc:2454] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64d72e7db6f44857b2902f8bec7a4810 in term 1.
I20260807 19:38:21.196826 22717 raft_consensus.cc:2454] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64d72e7db6f44857b2902f8bec7a4810 in term 1.
I20260807 19:38:21.197221 22360 leader_election.cc:302] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280e6bd8c951495ba5f75e6f9afe3b97, 64d72e7db6f44857b2902f8bec7a4810; no voters:
I20260807 19:38:21.197525 22818 leader_election.cc:288] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [CANDIDATE]: Term 1 election: Requested vote from peers 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66:45805), d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783)
I20260807 19:38:21.197759 22818 raft_consensus.cc:2781] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:21.197899 22818 raft_consensus.cc:686] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 LEADER]: Becoming Leader. State: Replica: 64d72e7db6f44857b2902f8bec7a4810, State: Running, Role: LEADER
I20260807 19:38:21.198217 22818 consensus_queue.cc:237] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } }
I20260807 19:38:21.210670 22265 catalog_manager.cc:5495] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65). New cstate: current_term: 1 leader_uuid: "64d72e7db6f44857b2902f8bec7a4810" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d72e7db6f44857b2902f8bec7a4810" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 } health_report { overall_health: UNKNOWN } } }
I20260807 19:38:21.277304 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:21.268041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.14.16.65:37685)
0807 19:38:21.268240 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:21.268246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:21.268266 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:38:21.268343 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:21.268348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:21.268453 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:21.268593 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:38:21.269380 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.270845 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.273875 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.274782 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.274926 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:21.275816 (+ 890us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:38:21.276011 (+ 195us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:38:21.276110 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:21.276769 (+ 659us) server_negotiation.cc:299] Negotiation successful
0807 19:38:21.276885 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 19:38:21.306589 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:21.299578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:40447 (local address 127.14.16.66:45805)
0807 19:38:21.299780 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:21.299791 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:21.299814 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:38:21.299912 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:21.299917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:21.300008 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:21.300120 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:38:21.300716 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.301917 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.304067 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.304863 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.305002 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:21.305576 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:38:21.305761 (+ 185us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:38:21.305865 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:21.306090 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 19:38:21.306180 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:38:21.306644 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:21.298940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:45805 (local address 127.14.16.65:40447)
0807 19:38:21.299610 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:21.299631 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:21.299784 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:21.300165 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:21.300171 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:21.300184 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:38:21.300514 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:21.300526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.302082 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.302088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:21.303878 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:21.303895 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.305055 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.305061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:21.305273 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:21.305361 (+ 88us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:38:21.305880 (+ 519us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:38:21.305894 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 19:38:21.306052 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 19:38:21.306202 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":285,"threads_started":1}
I20260807 19:38:21.318638 22717 raft_consensus.cc:1261] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Refusing update from remote peer 64d72e7db6f44857b2902f8bec7a4810: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:21.320322 22888 consensus_queue.cc:1035] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2af1c6e12ea424db07c1456c5dcdd5b" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:21.321446 22576 raft_consensus.cc:1261] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Refusing update from remote peer 64d72e7db6f44857b2902f8bec7a4810: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:21.322414 22888 consensus_queue.cc:1035] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "280e6bd8c951495ba5f75e6f9afe3b97" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 45805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:21.383641 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:21.372626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:44239 (local address 127.14.16.65:37685)
0807 19:38:21.372807 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:21.372812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:21.375577 (+ 2765us) server_negotiation.cc:407] Connection header received
0807 19:38:21.375859 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:21.375865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:21.375951 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:21.376019 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:38:21.376866 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.378054 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.381439 (+ 3385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.382106 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.382292 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:21.382778 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:38:21.382917 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:38:21.383019 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:21.383196 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 19:38:21.383326 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 19:38:21.383736 22900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:21.372582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:37685 (local address 127.14.16.66:44239)
0807 19:38:21.375471 (+ 2889us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:21.375496 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:21.375695 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:21.376227 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:21.376232 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:21.376245 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:38:21.376644 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:21.376659 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.378255 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.378278 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:21.381267 (+ 2989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:21.381282 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.382347 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.382354 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:21.382524 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:21.382573 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:38:21.383050 (+ 477us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:38:21.383062 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:38:21.383143 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 19:38:21.383315 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2662,"thread_start_us":408,"threads_started":1}
I20260807 19:38:21.427150 22264 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:21.427490 22264 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:21.430598 22264 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ad8a40fa4ff4d0fbf9cf26f900cd723: Sending DeleteTablet for 3 replicas of tablet c504c34517404586ac3ba51a03da9cb1
I20260807 19:38:21.431329 22407 tablet_service.cc:1514] Processing DeleteTablet for tablet c504c34517404586ac3ba51a03da9cb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:21 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:45414
I20260807 19:38:21.431625 22856 tablet_replica.cc:330] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810: stopping tablet replica
I20260807 19:38:21.431807 22856 raft_consensus.cc:2229] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:38:21.431875 22551 tablet_service.cc:1514] Processing DeleteTablet for tablet c504c34517404586ac3ba51a03da9cb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:21 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:44434
I20260807 19:38:21.432168 22856 raft_consensus.cc:2258] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:21.432222 22697 tablet_service.cc:1514] Processing DeleteTablet for tablet c504c34517404586ac3ba51a03da9cb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:21 UTC) from {username='oryx', principal='oryx/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:52226
I20260807 19:38:21.432520 22857 tablet_replica.cc:330] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97: stopping tablet replica
I20260807 19:38:21.432761 22857 raft_consensus.cc:2229] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:21.432765 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 22341
I20260807 19:38:21.432984 22857 raft_consensus.cc:2258] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:21.433276 22858 tablet_replica.cc:330] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b: stopping tablet replica
I20260807 19:38:21.433466 22858 raft_consensus.cc:2229] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:21.433697 22858 raft_consensus.cc:2258] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:21.434762 22856 ts_tablet_manager.cc:1905] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:21.435962 22857 ts_tablet_manager.cc:1905] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:21.439636 22856 ts_tablet_manager.cc:1918] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:38:21.439812 22856 log.cc:1196] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/c504c34517404586ac3ba51a03da9cb1
I20260807 19:38:21.440251 22856 ts_tablet_manager.cc:1939] T c504c34517404586ac3ba51a03da9cb1 P 64d72e7db6f44857b2902f8bec7a4810: Deleting consensus metadata
I20260807 19:38:21.440601 22858 ts_tablet_manager.cc:1905] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:21.441313 22251 catalog_manager.cc:4841] TS 64d72e7db6f44857b2902f8bec7a4810 (127.14.16.65:37685): tablet c504c34517404586ac3ba51a03da9cb1 (table test-table [id=cdd9e449bde34eb89a7bcb287ba77419]) successfully deleted
I20260807 19:38:21.442718 22857 ts_tablet_manager.cc:1918] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:38:21.442843 22857 log.cc:1196] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/c504c34517404586ac3ba51a03da9cb1
I20260807 19:38:21.447229 22858 ts_tablet_manager.cc:1918] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:38:21.447407 22858 log.cc:1196] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/c504c34517404586ac3ba51a03da9cb1
I20260807 19:38:21.447865 22858 ts_tablet_manager.cc:1939] T c504c34517404586ac3ba51a03da9cb1 P d2af1c6e12ea424db07c1456c5dcdd5b: Deleting consensus metadata
I20260807 19:38:21.449069 22249 catalog_manager.cc:4841] TS d2af1c6e12ea424db07c1456c5dcdd5b (127.14.16.67:41783): tablet c504c34517404586ac3ba51a03da9cb1 (table test-table [id=cdd9e449bde34eb89a7bcb287ba77419]) successfully deleted
I20260807 19:38:21.449396 22857 ts_tablet_manager.cc:1939] T c504c34517404586ac3ba51a03da9cb1 P 280e6bd8c951495ba5f75e6f9afe3b97: Deleting consensus metadata
I20260807 19:38:21.450568 22252 catalog_manager.cc:4841] TS 280e6bd8c951495ba5f75e6f9afe3b97 (127.14.16.66:45805): tablet c504c34517404586ac3ba51a03da9cb1 (table test-table [id=cdd9e449bde34eb89a7bcb287ba77419]) successfully deleted
Aug 07 19:38:21 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.16.65@KRBTEST.COM
I20260807 19:38:21.460629 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:21.435263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45444 (local address 127.14.16.65:37685)
0807 19:38:21.443453 (+ 8190us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:21.443460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:21.443482 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:38:21.443546 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:21.443551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:21.443645 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:21.443779 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:21.445689 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.446858 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.449000 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.449799 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.449901 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:21.454210 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:21.454224 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:21.454240 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:21.454277 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:21.457459 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:21.458162 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:21.458170 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:21.458172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:21.458300 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:21.458702 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:21.458711 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:21.458713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:21.459228 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:21.459636 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:21.459936 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 19:38:21.460179 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8056}
I20260807 19:38:21.461414 22474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:21.625360 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 22341
I20260807 19:38:21.651602 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 22483
Aug 07 19:38:21 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.16.66@KRBTEST.COM
I20260807 19:38:21.666105 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:21.652588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44458 (local address 127.14.16.66:45805)
0807 19:38:21.652784 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:21.652789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:21.652804 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:38:21.652860 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:21.652864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:21.652932 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:21.653017 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:21.654213 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.654893 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.656166 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.656587 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.656683 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:21.660778 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:21.660796 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:21.660821 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:21.660859 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:21.663939 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:21.664490 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:21.664496 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:21.664498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:21.664624 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:21.664941 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:21.664944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:21.664946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:21.665270 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:21.665496 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:21.665694 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 19:38:21.665856 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 19:38:21.666738 22619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:21.817436 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 22483
I20260807 19:38:21.847126 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 22628
Aug 07 19:38:21 dist-test-slave-07c2 krb5kdc[22220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.16.67@KRBTEST.COM
I20260807 19:38:21.863449 22905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:21.847929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.14.16.67:41783)
0807 19:38:21.848492 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:21.848496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:21.848511 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:38:21.848585 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:21.848589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:21.848656 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:21.848785 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:21.850122 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.850865 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.852192 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:21.852642 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:21.852774 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:21.857354 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:21.857370 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:21.857387 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:21.857425 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:21.860584 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:21.861285 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:21.861300 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:21.861302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:21.861431 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:21.861831 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:21.861835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:21.861837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:21.862192 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:21.862507 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:21.862811 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 19:38:21.863063 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":287,"threads_started":1}
I20260807 19:38:21.864144 22764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:21.988574 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 22628
I20260807 19:38:22.012951 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 22233
I20260807 19:38:22.014153 22306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:22.133317 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 22233
2026-08-07T19:38:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7152 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:38:22 dist-test-slave-07c2 krb5kdc[22910](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:38:22 dist-test-slave-07c2 krb5kdc[22910](info): set up 2 sockets
Aug 07 19:38:22 dist-test-slave-07c2 krb5kdc[22910](info): commencing operation
krb5kdc: starting...
W20260807 19:38:23.271042 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:38:23 dist-test-slave-07c2 krb5kdc[22910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:38:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:38:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:38:23.530869 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:39195
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:33569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:39195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:23.858724 22924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:23.859014 22924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:23.859181 22924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:23.866617 22924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:38:23.866742 22924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:23.866794 22924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:23.866880 22924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:38:23.866958 22924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:38:23.877142 22924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:33569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:39195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:39195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:23.878437 22924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:23.879804 22924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:23.891868 22929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:23.891811 22930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:23.891891 22932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:23.892757 22924 server_base.cc:1034] running on GCE node
I20260807 19:38:23.893534 22924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:23.894896 22924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:23.896130 22924 hybrid_clock.cc:648] HybridClock initialized: now 1786131503896074 us; error 65 us; skew 500 ppm
I20260807 19:38:23.896484 22924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T19:38:23Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1834 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:38:24 dist-test-slave-07c2 krb5kdc[22938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:38:24 dist-test-slave-07c2 krb5kdc[22938](info): set up 2 sockets
Aug 07 19:38:24 dist-test-slave-07c2 krb5kdc[22938](info): commencing operation
krb5kdc: starting...
W20260807 19:38:25.090862 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:38:25 dist-test-slave-07c2 krb5kdc[22938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:38:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:38:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:38:25.352707 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:44457
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:44457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:25.689090 22951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:25.689399 22951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:25.689591 22951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:25.696920 22951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:38:25.697039 22951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:25.697086 22951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:25.697176 22951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:38:25.697249 22951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:38:25.707154 22951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:44457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:44457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:25.708554 22951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:25.709868 22951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:25.721925 22956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:25.721997 22958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:25.721891 22960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:25.727891 22951 server_base.cc:1034] running on GCE node
I20260807 19:38:25.728695 22951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:25.730041 22951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:25.731253 22951 hybrid_clock.cc:648] HybridClock initialized: now 1786131505731207 us; error 34 us; skew 500 ppm
I20260807 19:38:25.731613 22951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:25 dist-test-slave-07c2 krb5kdc[22938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131505, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:25.738358 22951 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:38:25.740057 22951 webserver.cc:460] Webserver started at http://127.14.16.126:45845/ using document root <none> and password file <none>
I20260807 19:38:25.740805 22951 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:25.740947 22951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:25.741298 22951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:25.744374 22951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "11835bba90934cfba70c8172c59f5484"
format_stamp: "Formatted at 2026-08-07 19:38:25 on dist-test-slave-07c2"
server_key: "0fb2a7f0d9b07c041cbd1482f87160ef"
server_key_iv: "a9e833b625f355e293dede5481354793"
server_key_version: "encryptionkey@0"
I20260807 19:38:25.745141 22951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "11835bba90934cfba70c8172c59f5484"
format_stamp: "Formatted at 2026-08-07 19:38:25 on dist-test-slave-07c2"
server_key: "0fb2a7f0d9b07c041cbd1482f87160ef"
server_key_iv: "a9e833b625f355e293dede5481354793"
server_key_version: "encryptionkey@0"
I20260807 19:38:25.750358 22951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:38:25.753729 22967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:25.754997 22951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:38:25.755184 22951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "11835bba90934cfba70c8172c59f5484"
format_stamp: "Formatted at 2026-08-07 19:38:25 on dist-test-slave-07c2"
server_key: "0fb2a7f0d9b07c041cbd1482f87160ef"
server_key_iv: "a9e833b625f355e293dede5481354793"
server_key_version: "encryptionkey@0"
I20260807 19:38:25.755453 22951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:25.769105 22951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:25.773128 22951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:25.773438 22951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:25.792091 22951 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:44457
I20260807 19:38:25.792115 23020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:44457 every 8 connection(s)
I20260807 19:38:25.793509 22951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:38:25.797072 23021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:25.800393 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 22951
I20260807 19:38:25.800626 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:38:25.801481 14401 external_mini_cluster.cc:1428] Setting key 25988ddaf39a562e36973ea8d25b4ac5
I20260807 19:38:25.810294 23021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484: Bootstrap starting.
I20260807 19:38:25.814150 23021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:25.815510 23021 log.cc:824] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:38:25 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:38:25.818977 23021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484: No bootstrap required, opened a new log
I20260807 19:38:25.825553 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:25.803631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.14.16.126:44457)
0807 19:38:25.804455 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:25.804466 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:25.804508 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:38:25.806165 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:25.806186 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:25.806588 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:25.806904 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:25.808773 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:25.810061 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:25.811853 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:25.812530 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:25.812654 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:25.817790 (+ 5136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:25.817797 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:25.817811 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:25.817847 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:25.821387 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:25.822216 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:25.822229 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:25.822235 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:25.822401 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:25.822797 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:25.822800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:25.822802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:25.823395 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:25.823621 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:25.824117 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 19:38:25.824536 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":389,"threads_started":1}
I20260807 19:38:25.827822 23021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11835bba90934cfba70c8172c59f5484" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44457 } }
I20260807 19:38:25.828214 23021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:25.828341 23021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11835bba90934cfba70c8172c59f5484, State: Initialized, Role: FOLLOWER
I20260807 19:38:25.828951 23021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11835bba90934cfba70c8172c59f5484" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44457 } }
I20260807 19:38:25.829181 23021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:38:25.829344 23021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:38:25.829519 23021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:25.831725 23021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11835bba90934cfba70c8172c59f5484" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44457 } }
I20260807 19:38:25.832206 23021 leader_election.cc:302] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11835bba90934cfba70c8172c59f5484; no voters:
I20260807 19:38:25.832811 23021 leader_election.cc:288] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:38:25.832901 23026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:25.833770 23026 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 1 LEADER]: Becoming Leader. State: Replica: 11835bba90934cfba70c8172c59f5484, State: Running, Role: LEADER
I20260807 19:38:25.834312 23026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11835bba90934cfba70c8172c59f5484" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44457 } }
I20260807 19:38:25.835093 23021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:38:25.837359 23028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11835bba90934cfba70c8172c59f5484. Latest consensus state: current_term: 1 leader_uuid: "11835bba90934cfba70c8172c59f5484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11835bba90934cfba70c8172c59f5484" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44457 } } }
I20260807 19:38:25.837836 23028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:25.838977 23027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11835bba90934cfba70c8172c59f5484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11835bba90934cfba70c8172c59f5484" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44457 } } }
I20260807 19:38:25.839321 23027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:25.841289 23034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:38:25.846688 23034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:38:25.855751 23034 catalog_manager.cc:1347] Generated new cluster ID: 362453025e254c44b590097360e39718
I20260807 19:38:25.855893 23034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:38:25.867187 23034 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:38:25.869100 23034 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:38:25.888664 23034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484: Generated new TSK 0
I20260807 19:38:25.889550 23034 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:38:25.957001 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44457
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:26.291507 23047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:26.291867 23047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:26.291980 23047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:26.299029 23047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:26.299136 23047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:26.299247 23047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:38:26.310091 23047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:26.311395 23047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:26.312803 23047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:26.326224 23053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:26.326663 23055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:26.326694 23052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:26.327219 23047 server_base.cc:1034] running on GCE node
I20260807 19:38:26.327888 23047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:26.328783 23047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:26.330006 23047 hybrid_clock.cc:648] HybridClock initialized: now 1786131506329972 us; error 50 us; skew 500 ppm
I20260807 19:38:26.330319 23047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:26 dist-test-slave-07c2 krb5kdc[22938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131506, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:26.333351 23047 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:38:26.334698 23047 webserver.cc:460] Webserver started at http://127.14.16.65:35661/ using document root <none> and password file <none>
I20260807 19:38:26.335464 23047 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:26.335655 23047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:26.335995 23047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:26.338986 23047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "15db9a65d44341a482fbebf451927c4d"
format_stamp: "Formatted at 2026-08-07 19:38:26 on dist-test-slave-07c2"
server_key: "3352216a0cad7ef5f560fa60f5fd7117"
server_key_iv: "c705a0db31614546714814e10be18722"
server_key_version: "encryptionkey@0"
I20260807 19:38:26.339982 23047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "15db9a65d44341a482fbebf451927c4d"
format_stamp: "Formatted at 2026-08-07 19:38:26 on dist-test-slave-07c2"
server_key: "3352216a0cad7ef5f560fa60f5fd7117"
server_key_iv: "c705a0db31614546714814e10be18722"
server_key_version: "encryptionkey@0"
I20260807 19:38:26.345436 23047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:38:26.348825 23062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:26.350147 23047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:38:26.350276 23047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "15db9a65d44341a482fbebf451927c4d"
format_stamp: "Formatted at 2026-08-07 19:38:26 on dist-test-slave-07c2"
server_key: "3352216a0cad7ef5f560fa60f5fd7117"
server_key_iv: "c705a0db31614546714814e10be18722"
server_key_version: "encryptionkey@0"
I20260807 19:38:26.350430 23047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:26.373785 23047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:26.377547 23047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:26.377785 23047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:26.378680 23047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:26.380071 23047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:26.380146 23047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:26.380226 23047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:26.380277 23047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:26.414647 23047 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:37035
I20260807 19:38:26.414695 23178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:37035 every 8 connection(s)
I20260807 19:38:26.415993 23047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:38:26.424198 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 23047
I20260807 19:38:26.424396 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:38:26.425181 14401 external_mini_cluster.cc:1428] Setting key 19780b40268754dfdf4ad04adfd75b3d
Aug 07 19:38:26 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131506, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:38:26.438199 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:26.418565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:34589 (local address 127.14.16.126:44457)
0807 19:38:26.419011 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:26.419016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:26.420029 (+ 1013us) server_negotiation.cc:407] Connection header received
0807 19:38:26.421869 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:26.421875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:26.421969 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:26.422089 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:26.423617 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:26.424838 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:26.426744 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:26.427328 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:26.427468 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:26.431008 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:26.431017 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:26.431024 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:26.431068 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:26.434686 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:26.435494 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:26.435503 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:26.435509 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:26.435665 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:26.436104 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:26.436111 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:26.436117 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:26.436533 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:26.436690 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:26.437447 (+ 757us) server_negotiation.cc:299] Negotiation successful
0807 19:38:26.437702 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":233,"threads_started":1}
I20260807 19:38:26.438912 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:26.419106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:34589)
0807 19:38:26.419877 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:26.419916 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:26.421563 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:26.422321 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:26.422329 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:26.422845 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:26.423451 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:26.423470 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:26.424999 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:26.425003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:26.426584 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:26.426599 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:26.427541 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:26.427546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:26.427752 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:26.428462 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:26.428485 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:26.430791 (+ 2306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:26.434836 (+ 4045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:26.434842 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:26.434856 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:26.435384 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:26.435778 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:26.435782 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:26.435784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:26.435964 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:26.436698 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:26.436708 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:26.437039 (+ 331us) client_negotiation.cc:769] Sending connection context
0807 19:38:26.437276 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 19:38:26.437661 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":350,"threads_started":1}
I20260807 19:38:26.440654 23179 heartbeater.cc:347] Connected to a master server at 127.14.16.126:44457
I20260807 19:38:26.440989 23179 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:26.441848 23179 heartbeater.cc:511] Master 127.14.16.126:44457 requested a full tablet report, sending...
I20260807 19:38:26.443951 22985 ts_manager.cc:194] Registered new tserver with Master: 15db9a65d44341a482fbebf451927c4d (127.14.16.65:37035)
I20260807 19:38:26.445359 22985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:34589
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:38:26.482282 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44457
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:26.813441 23186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:26.813753 23186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:26.813918 23186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:26.821364 23186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:26.821489 23186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:26.821684 23186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:38:26.831836 23186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:26.833199 23186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:26.834498 23186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:26.848551 23191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:26.848592 23192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:26.848932 23194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:26.849588 23186 server_base.cc:1034] running on GCE node
I20260807 19:38:26.850251 23186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:26.851187 23186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:26.852416 23186 hybrid_clock.cc:648] HybridClock initialized: now 1786131506852364 us; error 39 us; skew 500 ppm
I20260807 19:38:26.852806 23186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:26 dist-test-slave-07c2 krb5kdc[22938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131506, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:26.856199 23186 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:38:26.857667 23186 webserver.cc:460] Webserver started at http://127.14.16.66:35103/ using document root <none> and password file <none>
I20260807 19:38:26.858363 23186 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:26.858527 23186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:26.858879 23186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:26.862231 23186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "5289f8273715457f8ceff8c22fe27754"
format_stamp: "Formatted at 2026-08-07 19:38:26 on dist-test-slave-07c2"
server_key: "5da01fe84542cdb8c2b41f1d1414e5d0"
server_key_iv: "d0adbd347072270eecb9b3b0bb9a6b54"
server_key_version: "encryptionkey@0"
I20260807 19:38:26.863021 23186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "5289f8273715457f8ceff8c22fe27754"
format_stamp: "Formatted at 2026-08-07 19:38:26 on dist-test-slave-07c2"
server_key: "5da01fe84542cdb8c2b41f1d1414e5d0"
server_key_iv: "d0adbd347072270eecb9b3b0bb9a6b54"
server_key_version: "encryptionkey@0"
I20260807 19:38:26.868147 23186 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 19:38:26.871793 23201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:26.873118 23186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:38:26.873309 23186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "5289f8273715457f8ceff8c22fe27754"
format_stamp: "Formatted at 2026-08-07 19:38:26 on dist-test-slave-07c2"
server_key: "5da01fe84542cdb8c2b41f1d1414e5d0"
server_key_iv: "d0adbd347072270eecb9b3b0bb9a6b54"
server_key_version: "encryptionkey@0"
I20260807 19:38:26.873510 23186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:26.904408 23186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:26.908649 23186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:26.908946 23186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:26.909943 23186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:26.911379 23186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:26.911481 23186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:26.911636 23186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:26.911718 23186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:26.946542 23186 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:37769
I20260807 19:38:26.946593 23317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:37769 every 8 connection(s)
I20260807 19:38:26.947940 23186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:38:26.951884 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 23186
I20260807 19:38:26.952207 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:38:26.952872 14401 external_mini_cluster.cc:1428] Setting key 778a35c26f68e792e89e35373e3ecffa
Aug 07 19:38:26 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131506, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:38:26.970319 23320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:26.950679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:33583 (local address 127.14.16.126:44457)
0807 19:38:26.951162 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:26.951168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:26.952696 (+ 1528us) server_negotiation.cc:407] Connection header received
0807 19:38:26.954931 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:26.954937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:26.955031 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:26.955200 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:26.956788 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:26.957547 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:26.959229 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:26.959793 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:26.959966 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:26.963564 (+ 3598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:26.963569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:26.963572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:26.963610 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:26.966739 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:26.967744 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:26.967748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:26.967751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:26.967890 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:26.968434 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:26.968443 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:26.968447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:26.968768 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:26.968887 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:26.969852 (+ 965us) server_negotiation.cc:299] Negotiation successful
0807 19:38:26.970029 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":228,"threads_started":1}
I20260807 19:38:26.971884 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:26.951340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:33583)
0807 19:38:26.952521 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:26.952559 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:26.954634 (+ 2075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:26.955470 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:26.955479 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:26.956041 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:26.956618 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:26.956635 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:26.957723 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:26.957727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:26.959079 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:26.959091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:26.960057 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:26.960067 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:26.960314 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:26.961032 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:26.961055 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:26.963270 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:26.966884 (+ 3614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:26.966895 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:26.966917 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:26.967594 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:26.968008 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:26.968015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:26.968022 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:26.968279 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:26.968905 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:26.968915 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:26.969376 (+ 461us) client_negotiation.cc:769] Sending connection context
0807 19:38:26.969725 (+ 349us) client_negotiation.cc:240] Negotiation successful
0807 19:38:26.970341 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"thread_start_us":702,"threads_started":1}
I20260807 19:38:26.973394 23318 heartbeater.cc:347] Connected to a master server at 127.14.16.126:44457
I20260807 19:38:26.973734 23318 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:26.974637 23318 heartbeater.cc:511] Master 127.14.16.126:44457 requested a full tablet report, sending...
I20260807 19:38:26.976223 22985 ts_manager.cc:194] Registered new tserver with Master: 5289f8273715457f8ceff8c22fe27754 (127.14.16.66:37769)
I20260807 19:38:26.977111 22985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:33583
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:38:27.012856 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44457
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:27.352612 23325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:27.352936 23325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:27.353111 23325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:27.360680 23325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:27.360833 23325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:27.361001 23325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:38:27.370586 23325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:27.371951 23325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:27.373267 23325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:27.386626 23330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:27.386564 23333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:27.386538 23331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:27.387688 23325 server_base.cc:1034] running on GCE node
I20260807 19:38:27.388341 23325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:27.389299 23325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:27.390537 23325 hybrid_clock.cc:648] HybridClock initialized: now 1786131507390489 us; error 59 us; skew 500 ppm
I20260807 19:38:27.390895 23325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:27 dist-test-slave-07c2 krb5kdc[22938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131507, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:27.394109 23325 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:38:27.396220 23325 webserver.cc:460] Webserver started at http://127.14.16.67:37177/ using document root <none> and password file <none>
I20260807 19:38:27.396919 23325 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:27.397068 23325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:27.397382 23325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:27.400380 23325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "ec1d8403b27541c9888f7d31699887a4"
format_stamp: "Formatted at 2026-08-07 19:38:27 on dist-test-slave-07c2"
server_key: "fb032e949af44adcc8e46281f3975dd0"
server_key_iv: "e84614ab4ed8e014cc3ce98833d5da49"
server_key_version: "encryptionkey@0"
I20260807 19:38:27.401180 23325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "ec1d8403b27541c9888f7d31699887a4"
format_stamp: "Formatted at 2026-08-07 19:38:27 on dist-test-slave-07c2"
server_key: "fb032e949af44adcc8e46281f3975dd0"
server_key_iv: "e84614ab4ed8e014cc3ce98833d5da49"
server_key_version: "encryptionkey@0"
I20260807 19:38:27.406306 23325 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:38:27.409901 23340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:27.411194 23325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:38:27.411409 23325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "ec1d8403b27541c9888f7d31699887a4"
format_stamp: "Formatted at 2026-08-07 19:38:27 on dist-test-slave-07c2"
server_key: "fb032e949af44adcc8e46281f3975dd0"
server_key_iv: "e84614ab4ed8e014cc3ce98833d5da49"
server_key_version: "encryptionkey@0"
I20260807 19:38:27.411616 23325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:27.429360 23325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:27.433787 23325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:27.434087 23325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:27.435187 23325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:27.436746 23325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:27.436822 23325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:27.436970 23325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:27.437021 23325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:27.448302 23179 heartbeater.cc:503] Master 127.14.16.126:44457 was elected leader, sending a full tablet report...
I20260807 19:38:27.472563 23325 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:35675
I20260807 19:38:27.472527 23457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:35675 every 8 connection(s)
I20260807 19:38:27.473878 23325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:38:27.480813 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 23325
I20260807 19:38:27.481009 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:38:27.481585 14401 external_mini_cluster.cc:1428] Setting key d12904beb0de60f6e2ce48abd9bd77fa
Aug 07 19:38:27 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131507, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:38:27.494644 23460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:27.476485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:47725 (local address 127.14.16.126:44457)
0807 19:38:27.477039 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:27.477044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:27.477970 (+ 926us) server_negotiation.cc:407] Connection header received
0807 19:38:27.479982 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:27.479988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:27.480083 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:27.480212 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:27.482339 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:27.483399 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:27.485086 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:27.485496 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:27.485631 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:27.489018 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:27.489028 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:27.489032 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:27.489064 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:27.491669 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:27.492363 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:27.492370 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:27.492374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:27.492501 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:27.492945 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:27.492951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:27.492954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:27.493227 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:27.493383 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:27.494042 (+ 659us) server_negotiation.cc:299] Negotiation successful
0807 19:38:27.494270 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":223,"threads_started":1}
I20260807 19:38:27.495889 23461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:27.477038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:47725)
0807 19:38:27.477796 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:27.477832 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:27.479709 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:27.480501 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:27.480515 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:27.481329 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:27.482151 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:27.482184 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:27.483549 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:27.483555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:27.484947 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:27.484959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:27.485735 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:27.485742 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:27.485966 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:27.486562 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:27.486583 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:27.488814 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:27.491808 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:27.491817 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:27.491837 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:27.492247 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:27.492621 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:27.492626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:27.492630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:27.492831 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:27.493357 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:27.493365 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:27.493693 (+ 328us) client_negotiation.cc:769] Sending connection context
0807 19:38:27.493957 (+ 264us) client_negotiation.cc:240] Negotiation successful
0807 19:38:27.494458 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":299,"threads_started":1}
I20260807 19:38:27.497514 23458 heartbeater.cc:347] Connected to a master server at 127.14.16.126:44457
I20260807 19:38:27.497853 23458 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:27.498579 23458 heartbeater.cc:511] Master 127.14.16.126:44457 requested a full tablet report, sending...
I20260807 19:38:27.499975 22984 ts_manager.cc:194] Registered new tserver with Master: ec1d8403b27541c9888f7d31699887a4 (127.14.16.67:35675)
I20260807 19:38:27.500847 22984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:47725
I20260807 19:38:27.509258 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab.
Entry for principal oryx/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab.
WARNING: no policy specified for oryx/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab.
Entry for principal oryx/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab.
WARNING: no policy specified for oryx/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab.
Entry for principal oryx/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab.
I20260807 19:38:27.681810 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 23047
Aug 07 19:38:27 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:38:27.698264 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:27.683089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43602 (local address 127.14.16.65:37035)
0807 19:38:27.683889 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:27.683897 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:27.683924 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:38:27.683990 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:27.683997 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:27.684216 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:27.684355 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:27.685666 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:27.686663 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:27.687931 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:27.688484 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:27.688599 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:27.692934 (+ 4335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:27.692945 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:27.692958 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:27.692994 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:27.695721 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:27.696334 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:27.696341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:27.696343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:27.696469 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:27.696845 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:27.696848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:27.696850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:27.697166 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:27.697382 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:27.697655 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 19:38:27.697861 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":440,"threads_started":1}
I20260807 19:38:27.699551 23174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:27.783645 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 23047
I20260807 19:38:27.802470 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 23186
Aug 07 19:38:27 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:38:27.818095 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:27.803450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57408 (local address 127.14.16.66:37769)
0807 19:38:27.804030 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:27.804037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:27.804062 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:38:27.804121 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:27.804127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:27.804342 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:27.804474 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:27.805636 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:27.806637 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:27.807858 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:27.808390 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:27.808489 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:27.812571 (+ 4082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:27.812582 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:27.812595 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:27.812630 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:27.815496 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:27.816167 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:27.816174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:27.816176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:27.816310 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:27.816665 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:27.816668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:27.816670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:27.816994 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:27.817234 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:27.817490 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 19:38:27.817684 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":309,"threads_started":1}
I20260807 19:38:27.819411 23313 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:27.902127 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 23186
I20260807 19:38:27.920377 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 23325
Aug 07 19:38:27 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:38:27.937165 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:27.921502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59440 (local address 127.14.16.67:35675)
0807 19:38:27.922059 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:27.922066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:27.922096 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:38:27.922181 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:27.922188 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:27.922394 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:27.922524 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:27.923878 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:27.924869 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:27.926168 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:27.926716 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:27.926826 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:27.931281 (+ 4455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:27.931350 (+ 69us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:27.931393 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:27.931440 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:27.934468 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:27.935165 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:27.935172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:27.935174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:27.935400 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:27.935770 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:27.935773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:27.935775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:27.936165 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:27.936400 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:27.936658 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 19:38:27.936856 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":275,"threads_started":1}
I20260807 19:38:27.938400 23453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:28.021435 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 23325
I20260807 19:38:28.040316 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 22951
I20260807 19:38:28.041620 23015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:28.147193 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 22951
I20260807 19:38:28.170120 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:44457
--webserver_interface=127.14.16.126
--webserver_port=45845
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:44457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:28.516930 23477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:28.517342 23477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:28.517516 23477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:28.525638 23477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:38:28.525801 23477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:28.525887 23477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:28.525970 23477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:38:28.526032 23477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:38:28.536696 23477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:44457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:44457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=45845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:28.538184 23477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:28.539674 23477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:28.552839 23482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:28.552915 23483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:28.552959 23485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:28.554279 23477 server_base.cc:1034] running on GCE node
I20260807 19:38:28.555263 23477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:28.557031 23477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:28.558277 23477 hybrid_clock.cc:648] HybridClock initialized: now 1786131508558232 us; error 48 us; skew 500 ppm
I20260807 19:38:28.558698 23477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:28 dist-test-slave-07c2 krb5kdc[22938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131508, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:28.562150 23477 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:38:28.563822 23477 webserver.cc:460] Webserver started at http://127.14.16.126:45845/ using document root <none> and password file <none>
I20260807 19:38:28.564567 23477 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:28.564736 23477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:28.570412 23477 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 19:38:28.573459 23492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:28.574950 23477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:38:28.575166 23477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "11835bba90934cfba70c8172c59f5484"
format_stamp: "Formatted at 2026-08-07 19:38:25 on dist-test-slave-07c2"
server_key: "0fb2a7f0d9b07c041cbd1482f87160ef"
server_key_iv: "a9e833b625f355e293dede5481354793"
server_key_version: "encryptionkey@0"
I20260807 19:38:28.576310 23477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:28.610456 23477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:28.615384 23477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:28.615692 23477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:28.634739 23477 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:44457
I20260807 19:38:28.634764 23545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:44457 every 8 connection(s)
I20260807 19:38:28.636399 23477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:38:28.637995 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 23477
I20260807 19:38:28.638901 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:37035
--local_ip_for_outbound_sockets=127.14.16.65
--tserver_master_addrs=127.14.16.126:44457
--webserver_port=35661
--webserver_interface=127.14.16.65
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.16.65
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:38:28.643831 23546 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:38:28.647117 23546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484: Bootstrap starting.
I20260807 19:38:28.673936 23546 log.cc:824] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:28.685487 23546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:38:28.686165 23546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484: Bootstrap complete.
I20260807 19:38:28.697170 23546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11835bba90934cfba70c8172c59f5484" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44457 } }
I20260807 19:38:28.698009 23546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11835bba90934cfba70c8172c59f5484, State: Initialized, Role: FOLLOWER
I20260807 19:38:28.698669 23546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11835bba90934cfba70c8172c59f5484" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44457 } }
I20260807 19:38:28.698912 23546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:38:28.699044 23546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:38:28.699235 23546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:38:28.703140 23546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11835bba90934cfba70c8172c59f5484" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44457 } }
I20260807 19:38:28.703653 23546 leader_election.cc:302] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11835bba90934cfba70c8172c59f5484; no voters:
I20260807 19:38:28.704337 23546 leader_election.cc:288] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:38:28.704480 23551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:38:28.705811 23551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [term 2 LEADER]: Becoming Leader. State: Replica: 11835bba90934cfba70c8172c59f5484, State: Running, Role: LEADER
I20260807 19:38:28.706396 23551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11835bba90934cfba70c8172c59f5484" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44457 } }
I20260807 19:38:28.706638 23546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:38:28.709247 23553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11835bba90934cfba70c8172c59f5484. Latest consensus state: current_term: 2 leader_uuid: "11835bba90934cfba70c8172c59f5484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11835bba90934cfba70c8172c59f5484" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44457 } } }
I20260807 19:38:28.709707 23553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:28.710269 23552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "11835bba90934cfba70c8172c59f5484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11835bba90934cfba70c8172c59f5484" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44457 } } }
I20260807 19:38:28.710619 23552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:28.720594 23556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:38:28.728116 23556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:38:28.730041 23556 catalog_manager.cc:1259] Loaded cluster ID: 362453025e254c44b590097360e39718
I20260807 19:38:28.730180 23556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:38:28.735565 23556 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:38:28.737743 23556 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 11835bba90934cfba70c8172c59f5484: Loaded TSK: 0
I20260807 19:38:28.739727 23556 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:38:29.015862 23548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:29.016223 23548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:29.016386 23548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:29.023886 23548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:29.024048 23548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:29.024230 23548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:38:29.034327 23548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.65.keytab
--principal=oryx/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:37035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=35661
--tserver_master_addrs=127.14.16.126:44457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:29.035885 23548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:29.037273 23548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:29.052484 23574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:29.052637 23573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:29.052752 23576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:29.054292 23548 server_base.cc:1034] running on GCE node
I20260807 19:38:29.055022 23548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:29.056164 23548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:29.057404 23548 hybrid_clock.cc:648] HybridClock initialized: now 1786131509057336 us; error 52 us; skew 500 ppm
I20260807 19:38:29.057785 23548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:29 dist-test-slave-07c2 krb5kdc[22938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131509, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:29.061836 23548 init.cc:377] Logged in from keytab as oryx/127.14.16.65@KRBTEST.COM (short username oryx)
I20260807 19:38:29.063733 23548 webserver.cc:460] Webserver started at http://127.14.16.65:35661/ using document root <none> and password file <none>
I20260807 19:38:29.064704 23548 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:29.064862 23548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:29.072141 23548 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 19:38:29.075773 23583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:29.077567 23548 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:38:29.077762 23548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "15db9a65d44341a482fbebf451927c4d"
format_stamp: "Formatted at 2026-08-07 19:38:26 on dist-test-slave-07c2"
server_key: "3352216a0cad7ef5f560fa60f5fd7117"
server_key_iv: "c705a0db31614546714814e10be18722"
server_key_version: "encryptionkey@0"
I20260807 19:38:29.079378 23548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:29.096200 23548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:29.101845 23548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:29.102156 23548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:29.103488 23548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:29.105120 23548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:29.105230 23548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:29.105327 23548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:29.105407 23548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:29.141937 23548 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:37035
I20260807 19:38:29.141978 23698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:37035 every 8 connection(s)
I20260807 19:38:29.143755 23548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:38:29.144340 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 23548
I20260807 19:38:29.145205 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:37769
--local_ip_for_outbound_sockets=127.14.16.66
--tserver_master_addrs=127.14.16.126:44457
--webserver_port=35103
--webserver_interface=127.14.16.66
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.16.66
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:38:29 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:29 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:38:29.506735 23703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:29.507120 23703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:29.507396 23703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:29.515719 23703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:29.515843 23703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:29.516023 23703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:38:29.526466 23703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.66.keytab
--principal=oryx/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:37769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=35103
--tserver_master_addrs=127.14.16.126:44457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:29.527976 23703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:29.529364 23703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:29.546867 23710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:29.547051 23712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:29.546839 23709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:29.549156 23703 server_base.cc:1034] running on GCE node
I20260807 19:38:29.549871 23703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:29.550977 23703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:29.554384 23703 hybrid_clock.cc:648] HybridClock initialized: now 1786131509554307 us; error 114 us; skew 500 ppm
I20260807 19:38:29.554827 23703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:29 dist-test-slave-07c2 krb5kdc[22938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131509, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:29.559122 23703 init.cc:377] Logged in from keytab as oryx/127.14.16.66@KRBTEST.COM (short username oryx)
I20260807 19:38:29.561113 23703 webserver.cc:460] Webserver started at http://127.14.16.66:35103/ using document root <none> and password file <none>
I20260807 19:38:29.561895 23703 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:29.562062 23703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:29.568526 23703 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:38:29.571614 23720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:29.577636 23703 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.001s
I20260807 19:38:29.577912 23703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "5289f8273715457f8ceff8c22fe27754"
format_stamp: "Formatted at 2026-08-07 19:38:26 on dist-test-slave-07c2"
server_key: "5da01fe84542cdb8c2b41f1d1414e5d0"
server_key_iv: "d0adbd347072270eecb9b3b0bb9a6b54"
server_key_version: "encryptionkey@0"
I20260807 19:38:29.579717 23703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:29.596517 23703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:29.601191 23703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:29.601491 23703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:29.602639 23703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:29.604223 23703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:29.604353 23703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:29.604496 23703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:29.604588 23703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:29.648775 23703 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:37769
I20260807 19:38:29.648833 23838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:37769 every 8 connection(s)
I20260807 19:38:29.650228 23703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:38:29.656605 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 23703
I20260807 19:38:29.657560 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:35675
--local_ip_for_outbound_sockets=127.14.16.67
--tserver_master_addrs=127.14.16.126:44457
--webserver_port=37177
--webserver_interface=127.14.16.67
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.16.67
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:38:29 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:29 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:38:30.359500 23843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:30.359912 23843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:30.360106 23843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:30.367929 23843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:30.368078 23843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:30.368265 23843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:38:30.378903 23843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/krb5kdc/oryx_127.14.16.67.keytab
--principal=oryx/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:35675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=37177
--tserver_master_addrs=127.14.16.126:44457
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:30.380460 23843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:30.381839 23843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:30.401168 23849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:30.401208 23850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:30.402712 23852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:30.403116 23843 server_base.cc:1034] running on GCE node
I20260807 19:38:30.403857 23843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:30.404920 23843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:30.406630 23843 hybrid_clock.cc:648] HybridClock initialized: now 1786131510406533 us; error 86 us; skew 500 ppm
I20260807 19:38:30.406996 23843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:30 dist-test-slave-07c2 krb5kdc[22938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131510, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:30.410936 23843 init.cc:377] Logged in from keytab as oryx/127.14.16.67@KRBTEST.COM (short username oryx)
I20260807 19:38:30.414260 23843 webserver.cc:460] Webserver started at http://127.14.16.67:37177/ using document root <none> and password file <none>
I20260807 19:38:30.415000 23843 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:30.415199 23843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:30.423035 23843 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 19:38:30.426760 23859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:30.428797 23843 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:38:30.428995 23843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "ec1d8403b27541c9888f7d31699887a4"
format_stamp: "Formatted at 2026-08-07 19:38:27 on dist-test-slave-07c2"
server_key: "fb032e949af44adcc8e46281f3975dd0"
server_key_iv: "e84614ab4ed8e014cc3ce98833d5da49"
server_key_version: "encryptionkey@0"
I20260807 19:38:30.430341 23843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:30.456036 23843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:30.462004 23843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:30.462316 23843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:30.463723 23843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:30.465874 23843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:30.466007 23843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:30.466111 23843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:30.466218 23843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:30.533144 23843 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:35675
I20260807 19:38:30.533470 23976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:35675 every 8 connection(s)
I20260807 19:38:30.534978 23843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:38:30.539987 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 23843
W20260807 19:38:30.553857 23702 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.392s user 0.000s sys 0.014s
W20260807 19:38:30.557361 23701 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:29.147099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:43521 (local address 127.14.16.126:44457)
0807 19:38:29.147925 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:29.147942 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:29.154164 (+ 6222us) server_negotiation.cc:407] Connection header received
0807 19:38:29.154508 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:29.154520 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:29.154855 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:29.155235 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:29.156845 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:29.158066 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:29.159764 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:29.160419 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:29.160636 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:30.556059 (+1395423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:43521: BlockingRecv error: recv got EOF from 127.14.16.65:43521 (error 108)
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":307,"threads_started":1}
I20260807 19:38:30.559232 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:29.147091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:43521)
0807 19:38:29.151947 (+ 4856us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:29.151999 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:29.153833 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:29.155404 (+ 1571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:29.155417 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:29.156089 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:29.156616 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:29.156641 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:29.158250 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:29.158260 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:29.159606 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:29.159619 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:29.160682 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:29.160690 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:29.160970 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:29.161710 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:29.161732 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:30.555037 (+1393305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4540,"thread_start_us":4326,"threads_started":1}
W20260807 19:38:30.560714 23699 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:44457 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.16.126:44457: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Aug 07 19:38:30 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:30 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:30 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:30 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:30.579402 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:30.561556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:48741)
0807 19:38:30.562820 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:30.562841 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:30.562958 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:30.565583 (+ 2625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:30.565588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:30.566083 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:30.566368 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:30.566382 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.568085 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.568091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:30.570967 (+ 2876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:30.570984 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.572666 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.572675 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:30.572844 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:30.574498 (+ 1654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:30.574531 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:30.578957 (+ 4426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1134,"thread_start_us":3211,"threads_started":1}
W20260807 19:38:30.582798 23982 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:30.562076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:48741 (local address 127.14.16.126:44457)
0807 19:38:30.562858 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:30.562866 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:30.562886 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:38:30.565259 (+ 2373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:30.565269 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:30.565382 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:30.565555 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:30.566570 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.567914 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.571126 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.572428 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.573142 (+ 714us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:30.580293 (+ 7151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:48741: BlockingRecv error: recv got EOF from 127.14.16.65:48741 (error 108)
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":677}
I20260807 19:38:30.594082 23701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:30.542707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45736 (local address 127.14.16.126:44457)
0807 19:38:30.557894 (+ 15187us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:30.557904 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:30.565071 (+ 7167us) server_negotiation.cc:407] Connection header received
0807 19:38:30.565144 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:30.565149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:30.565235 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:30.565329 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:30.568735 (+ 3406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.569875 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.573636 (+ 3761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.574241 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.574367 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:30.582624 (+ 8257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:30.582639 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:30.582668 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:30.582724 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:30.588030 (+ 5306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:30.590264 (+ 2234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:30.590277 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:30.590290 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:30.590542 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:30.591195 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:30.591202 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:30.591209 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:30.592037 (+ 828us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:30.592388 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:30.593048 (+ 660us) server_negotiation.cc:299] Negotiation successful
0807 19:38:30.593537 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15042,"thread_start_us":18874,"threads_started":1}
Aug 07 19:38:30 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:30 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:30.605813 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:30.580405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:40961)
0807 19:38:30.580771 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:30.580791 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:30.580908 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:30.584273 (+ 3365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:30.584283 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:30.585991 (+ 1708us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:30.586351 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:30.586371 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.590264 (+ 3893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.590275 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:30.595144 (+ 4869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:30.595174 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.596461 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.596470 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:30.596700 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:30.597560 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:30.597624 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:30.605326 (+ 7702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248}
W20260807 19:38:30.606388 23984 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:30.580732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:40961 (local address 127.14.16.126:44457)
0807 19:38:30.583067 (+ 2335us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:30.583077 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:30.583121 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:38:30.583210 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:30.583218 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:30.583364 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:30.583526 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:30.586653 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.589968 (+ 3315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.595450 (+ 5482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.596271 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.596403 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:30.605853 (+ 9450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:40961: BlockingRecv error: recv got EOF from 127.14.16.65:40961 (error 108)
Metrics: {"server-negotiator.queue_time_us":2134,"thread_start_us":1908,"threads_started":1}
W20260807 19:38:30.606637 23699 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:38:30.837949 23841 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.159s user 0.002s sys 0.013s
W20260807 19:38:30.839551 23842 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:29.655002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:59693 (local address 127.14.16.126:44457)
0807 19:38:29.662789 (+ 7787us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:29.662800 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:29.667353 (+ 4553us) server_negotiation.cc:407] Connection header received
0807 19:38:29.671326 (+ 3973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:29.671336 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:29.671459 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:29.671596 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:29.674698 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:29.675673 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:29.677321 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:29.677744 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:29.677893 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:30.839179 (+1161286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:59693: BlockingRecv error: recv got EOF from 127.14.16.66:59693 (error 108)
Metrics: {"server-negotiator.queue_time_us":7552,"thread_start_us":8998,"threads_started":1}
I20260807 19:38:30.840050 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:29.654201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:59693)
0807 19:38:29.664315 (+ 10114us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:29.664406 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:29.670782 (+ 6376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:29.672025 (+ 1243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:29.672051 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:29.673234 (+ 1183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:29.674401 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:29.674437 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:29.675860 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:29.675866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:29.677138 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:29.677151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:29.678024 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:29.678032 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:29.678276 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:29.679004 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:29.679033 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:30.838908 (+1159875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":9593,"thread_start_us":13042,"threads_started":1}
W20260807 19:38:30.841266 23839 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:44457 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.16.126:44457: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Aug 07 19:38:30 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:30 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:30.859503 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:30.842096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:53503)
0807 19:38:30.842409 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:30.842426 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:30.842517 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:30.842913 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:30.842916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:30.843255 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:30.843506 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:30.843517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.845613 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.845616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:30.846538 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:30.846559 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.847881 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.847886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:30.848010 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:30.855104 (+ 7094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:30.855122 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:30.859235 (+ 4113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169}
W20260807 19:38:30.859987 23842 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:30.842202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:53503 (local address 127.14.16.126:44457)
0807 19:38:30.842475 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:30.842485 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:30.842515 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:38:30.842633 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:30.842641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:30.842753 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:30.842901 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:30.843670 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.845423 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.846692 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.847693 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.847870 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:30.859626 (+ 11756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:53503: BlockingRecv error: recv got EOF from 127.14.16.66:53503 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 19:38:30 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:30 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:38:30.871516 23842 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:30.860648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:47719 (local address 127.14.16.126:44457)
0807 19:38:30.860871 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:30.860880 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:30.860906 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:38:30.860977 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:30.860985 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:30.861104 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:30.861257 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:30.861881 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.863578 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.864834 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.865848 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.866023 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:30.871216 (+ 5193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:47719: BlockingRecv error: recv got EOF from 127.14.16.66:47719 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:38:30.871531 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:30.860218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:47719)
0807 19:38:30.860358 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:30.860381 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:30.860459 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:30.861290 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:30.861293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:30.861581 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:30.861730 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:30.861738 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.863748 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.863752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:30.864678 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:30.864689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.866106 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.866118 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:30.866258 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:30.866871 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:30.866888 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:30.871048 (+ 4160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 19:38:30.871953 23839 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 19:38:31 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:31 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:31.617866 23986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:31.607173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:51507)
0807 19:38:31.607941 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:31.607963 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:31.608086 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:31.608532 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:31.608546 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:31.609105 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:31.609294 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:31.609305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.610258 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.610263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:31.611860 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:31.611876 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.612607 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.612614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:31.612811 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:31.613501 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:31.613524 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:31.617623 (+ 4099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":501,"threads_started":1}
W20260807 19:38:31.618134 23987 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:31.607270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:51507 (local address 127.14.16.126:44457)
0807 19:38:31.608262 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:31.608266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:31.608282 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:38:31.608326 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:31.608329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:31.608384 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:31.608471 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:31.609434 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.610125 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.612004 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.612430 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.612524 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:31.617872 (+ 5348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:51507: BlockingRecv error: recv got EOF from 127.14.16.65:51507 (error 108)
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":734,"threads_started":1}
W20260807 19:38:31.688766 23981 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.115s user 0.000s sys 0.020s
W20260807 19:38:31.689972 23980 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:30.545780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:41767 (local address 127.14.16.126:44457)
0807 19:38:30.561861 (+ 16081us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:30.561868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:30.561888 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:38:30.561967 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:30.561977 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:30.562070 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:30.562214 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:30.567401 (+ 5187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.568653 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.571117 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.571748 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.571890 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:31.689748 (+1117858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:41767: BlockingRecv error: recv got EOF from 127.14.16.67:41767 (error 108)
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":15965,"thread_start_us":16819,"threads_started":1}
I20260807 19:38:31.690546 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:30.544815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:41767)
0807 19:38:30.558364 (+ 13549us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:30.558428 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:30.561026 (+ 2598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:30.565446 (+ 4420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:30.565459 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:30.566302 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:30.567166 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:30.567195 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.568885 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.568890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:30.570937 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:30.570954 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:30.572007 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:30.572017 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:30.572335 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:30.573344 (+ 1009us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:30.573375 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:31.689569 (+1116194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":13161,"thread_start_us":17873,"threads_started":1}
W20260807 19:38:31.691550 23977 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:44457 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.16.126:44457: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Aug 07 19:38:31 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:31 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:31.701297 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:31.692178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:35577)
0807 19:38:31.692383 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:31.692400 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:31.692493 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:31.692764 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:31.692767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:31.693117 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:31.693292 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:31.693302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.694284 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.694287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:31.695215 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:31.695236 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.695998 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.696003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:31.696130 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:31.696785 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:31.696804 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:31.700972 (+ 4168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 19:38:31.701426 23980 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:31.692280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:35577 (local address 127.14.16.126:44457)
0807 19:38:31.692477 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:31.692481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:31.692498 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:38:31.692577 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:31.692580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:31.692638 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:31.692722 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:31.693416 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.694153 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.695426 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.695842 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.695939 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:31.701181 (+ 5242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:35577: BlockingRecv error: recv got EOF from 127.14.16.67:35577 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 07 19:38:31 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:31 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:31.710623 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:31.702036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:50237)
0807 19:38:31.702230 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:31.702249 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:31.702364 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:31.702729 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:31.702734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:31.703080 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:31.703265 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:31.703275 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.704327 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.704333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:31.705322 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:31.705334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.706093 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.706099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:31.706306 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:31.706898 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:31.706915 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:31.710409 (+ 3494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 19:38:31.710947 23980 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:31.702132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:50237 (local address 127.14.16.126:44457)
0807 19:38:31.702329 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:31.702335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:31.702352 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:38:31.702496 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:31.702501 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:31.702575 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:31.702716 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:31.703453 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.704176 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.705470 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.705936 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.706079 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:31.710628 (+ 4549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:50237: BlockingRecv error: recv got EOF from 127.14.16.67:50237 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 19:38:31.710996 23977 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 19:38:31 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:31 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:31.882251 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:31.872755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:51887)
0807 19:38:31.873339 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:31.873355 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:31.873540 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:31.873958 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:31.873964 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:31.874551 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:31.874725 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:31.874735 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.875892 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.875896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:31.876853 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:31.876866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.877609 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.877613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:31.877741 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:31.878335 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:31.878353 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:31.881997 (+ 3644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":334,"threads_started":1}
W20260807 19:38:31.882440 23980 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:31.872857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:51887 (local address 127.14.16.126:44457)
0807 19:38:31.873142 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:31.873151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:31.873456 (+ 305us) server_negotiation.cc:407] Connection header received
0807 19:38:31.873694 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:31.873698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:31.873801 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:31.873889 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:31.874874 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.875755 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.877018 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:31.877445 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:31.877546 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:31.882190 (+ 4644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:51887: BlockingRecv error: recv got EOF from 127.14.16.66:51887 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 07 19:38:32 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:32 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:32.628134 23989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:32.618894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:40411)
0807 19:38:32.619433 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:32.619449 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:32.619587 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:32.619901 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:32.619905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:32.620299 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:32.620485 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:32.620493 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:32.621519 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:32.621523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:32.622470 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:32.622484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:32.623307 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:32.623314 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:32.623505 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:32.624133 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:32.624152 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:32.627882 (+ 3730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":283,"threads_started":1}
W20260807 19:38:32.628337 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:32.618947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:40411 (local address 127.14.16.126:44457)
0807 19:38:32.619462 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:32.619467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:32.619523 (+ 56us) server_negotiation.cc:407] Connection header received
0807 19:38:32.619690 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:32.619694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:32.619757 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:32.619849 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:32.620641 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:32.621374 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:32.622642 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:32.623118 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:32.623263 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:32.628070 (+ 4807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:40411: BlockingRecv error: recv got EOF from 127.14.16.65:40411 (error 108)
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":289,"threads_started":1}
Aug 07 19:38:32 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:32 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:32.720528 23991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:32.711643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:49787)
0807 19:38:32.712257 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:32.712278 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:32.712393 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:32.712682 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:32.712685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:32.713137 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:32.713285 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:32.713294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:32.714372 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:32.714376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:32.715359 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:32.715369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:32.716054 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:32.716059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:32.716190 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:32.716751 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:32.716769 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:32.720218 (+ 3449us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":349,"threads_started":1}
W20260807 19:38:32.720963 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:32.711694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:49787 (local address 127.14.16.126:44457)
0807 19:38:32.711883 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:32.711888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:32.712356 (+ 468us) server_negotiation.cc:407] Connection header received
0807 19:38:32.712490 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:32.712494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:32.712554 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:32.712633 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:32.713410 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:32.714237 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:32.715481 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:32.715899 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:32.715994 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:32.720467 (+ 4473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:49787: BlockingRecv error: recv got EOF from 127.14.16.67:49787 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 19:38:32 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:32 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:32.893147 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:32.883256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:36369)
0807 19:38:32.883864 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:32.883881 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:32.883993 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:32.884310 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:32.884313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:32.884820 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:32.885073 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:32.885083 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:32.886260 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:32.886264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:32.887225 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:32.887238 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:32.887978 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:32.887983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:32.888118 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:32.888739 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:32.888758 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:32.892775 (+ 4017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":374,"threads_started":1}
W20260807 19:38:32.893275 23990 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:32.883388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:36369 (local address 127.14.16.126:44457)
0807 19:38:32.883560 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:32.883565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:32.883952 (+ 387us) server_negotiation.cc:407] Connection header received
0807 19:38:32.884088 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:32.884092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:32.884168 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:32.884265 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:32.885202 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:32.886093 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:32.887422 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:32.887819 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:32.887925 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:32.892969 (+ 5044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:36369: BlockingRecv error: recv got EOF from 127.14.16.66:36369 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 19:38:33 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:33 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:33.638128 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:33.629059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:57195)
0807 19:38:33.629611 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:33.629627 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:33.629734 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:33.630168 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:33.630171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:33.630530 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:33.630704 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:33.630712 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:33.631732 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:33.631736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:33.632640 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:33.632652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:33.633318 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:33.633322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:33.633440 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:33.634004 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:33.634020 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:33.637885 (+ 3865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":267,"threads_started":1}
W20260807 19:38:33.638412 23994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:33.629252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:57195 (local address 127.14.16.126:44457)
0807 19:38:33.629904 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:33.629908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:33.629923 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:38:33.629965 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:33.629968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:33.630034 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:33.630120 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:33.630852 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:33.631606 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:33.632778 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:33.633186 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:33.633275 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:33.638138 (+ 4863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:57195: BlockingRecv error: recv got EOF from 127.14.16.65:57195 (error 108)
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":423,"threads_started":1}
Aug 07 19:38:33 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:33 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:33.731640 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:33.721603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:42845)
0807 19:38:33.722131 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:33.722153 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:33.722339 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:33.722707 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:33.722711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:33.723400 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:33.723668 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:33.723684 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:33.724967 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:33.724971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:33.725974 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:33.725986 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:33.726749 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:33.726753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:33.726886 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:33.727508 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:33.727527 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:33.731256 (+ 3729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":315,"threads_started":1}
W20260807 19:38:33.731734 23994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:33.721722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:42845 (local address 127.14.16.126:44457)
0807 19:38:33.721998 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:33.722003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:33.722247 (+ 244us) server_negotiation.cc:407] Connection header received
0807 19:38:33.722463 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:33.722466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:33.722530 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:33.722612 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:33.723847 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:33.724831 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:33.726147 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:33.726597 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:33.726709 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:33.731498 (+ 4789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:42845: BlockingRecv error: recv got EOF from 127.14.16.67:42845 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 07 19:38:33 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:33 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:33.903633 23996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:33.894237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:43145)
0807 19:38:33.894898 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:33.894914 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:33.895086 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:33.895475 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:33.895480 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:33.895898 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:33.896068 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:33.896077 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:33.897157 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:33.897163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:33.898070 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:33.898081 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:33.898828 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:33.898833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:33.899073 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:33.899739 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:33.899757 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:33.903269 (+ 3512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":378,"threads_started":1}
W20260807 19:38:33.903789 23994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:33.894323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:43145 (local address 127.14.16.126:44457)
0807 19:38:33.894687 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:33.894693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:33.895008 (+ 315us) server_negotiation.cc:407] Connection header received
0807 19:38:33.895235 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:33.895239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:33.895328 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:33.895438 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:33.896226 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:33.896999 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:33.898218 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:33.898665 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:33.898808 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:33.903525 (+ 4717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:43145: BlockingRecv error: recv got EOF from 127.14.16.66:43145 (error 108)
Metrics: {"server-negotiator.queue_time_us":203}
Aug 07 19:38:34 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:34 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:34.649469 23997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:34.639136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:34329)
0807 19:38:34.639742 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:34.639765 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:34.639899 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:34.640374 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:34.640380 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:34.640794 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:34.640965 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:34.640974 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:34.642219 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:34.642225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:34.643357 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:34.643371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:34.644110 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:34.644119 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:34.644338 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:34.644974 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:34.644993 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:34.649112 (+ 4119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":335,"threads_started":1}
W20260807 19:38:34.649632 23998 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:34.639228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:34329 (local address 127.14.16.126:44457)
0807 19:38:34.640068 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:34.640072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:34.640086 (+ 14us) server_negotiation.cc:407] Connection header received
0807 19:38:34.640128 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:34.640131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:34.640186 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:34.640323 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:34.641146 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:34.642078 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:34.643510 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:34.643941 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:34.644078 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:34.649359 (+ 5281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:34329: BlockingRecv error: recv got EOF from 127.14.16.65:34329 (error 108)
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":559,"threads_started":1}
Aug 07 19:38:34 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:34 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:34.742010 23999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:34.732750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:51489)
0807 19:38:34.733348 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:34.733369 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:34.733481 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:34.733775 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:34.733779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:34.734131 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:34.734306 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:34.734315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:34.735407 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:34.735411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:34.736451 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:34.736463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:34.737166 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:34.737174 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:34.737304 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:34.737926 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:34.737944 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:34.741748 (+ 3804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":310,"threads_started":1}
W20260807 19:38:34.742291 23998 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:34.732844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:51489 (local address 127.14.16.126:44457)
0807 19:38:34.733055 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:34.733060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:34.733446 (+ 386us) server_negotiation.cc:407] Connection header received
0807 19:38:34.733571 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:34.733575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:34.733636 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:34.733715 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:34.734465 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:34.735264 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:34.736573 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:34.737016 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:34.737103 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:34.741966 (+ 4863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:51489: BlockingRecv error: recv got EOF from 127.14.16.67:51489 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 19:38:34 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:34 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:34.914377 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:34.904802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:57445)
0807 19:38:34.905391 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:34.905409 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:34.905549 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:34.905912 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:34.905916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:34.906330 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:34.906513 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:34.906523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:34.907715 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:34.907719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:34.908881 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:34.908894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:34.909667 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:34.909671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:34.909796 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:34.910406 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:34.910425 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:34.914097 (+ 3672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":339,"threads_started":1}
W20260807 19:38:34.914660 23998 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:34.904901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:57445 (local address 127.14.16.126:44457)
0807 19:38:34.905177 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:34.905183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:34.905499 (+ 316us) server_negotiation.cc:407] Connection header received
0807 19:38:34.905683 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:34.905687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:34.905768 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:34.905846 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:34.906643 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:34.907550 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:34.909045 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:34.909520 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:34.909628 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:34.914367 (+ 4739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:57445: BlockingRecv error: recv got EOF from 127.14.16.66:57445 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 07 19:38:35 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:35 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:35.660022 24001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:35.650525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:54843)
0807 19:38:35.651064 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:35.651087 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:35.651221 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:35.651735 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:35.651741 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:35.652170 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:35.652327 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:35.652337 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:35.653422 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:35.653427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:35.654868 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:35.654882 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:35.655593 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:35.655597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:35.655719 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:35.656361 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:35.656380 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:35.659802 (+ 3422us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":263,"threads_started":1}
W20260807 19:38:35.660243 24002 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:35.650618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:54843 (local address 127.14.16.126:44457)
0807 19:38:35.651430 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:35.651434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:35.651450 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:38:35.651496 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:35.651499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:35.651556 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:35.651643 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:35.652462 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:35.653274 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:35.654999 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:35.655464 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:35.655559 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:35.659986 (+ 4427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:54843: BlockingRecv error: recv got EOF from 127.14.16.65:54843 (error 108)
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":536,"threads_started":1}
Aug 07 19:38:35 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:35 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:35.752262 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:35.743138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:41343)
0807 19:38:35.743644 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:35.743665 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:35.743805 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:35.744067 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:35.744070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:35.744471 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:35.744636 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:35.744645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:35.745647 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:35.745650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:35.746661 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:35.746673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:35.747422 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:35.747427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:35.747613 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:35.748333 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:35.748357 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:35.752005 (+ 3648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":285,"threads_started":1}
W20260807 19:38:35.752449 24002 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:35.743241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:41343 (local address 127.14.16.126:44457)
0807 19:38:35.743496 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:35.743502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:35.743747 (+ 245us) server_negotiation.cc:407] Connection header received
0807 19:38:35.743907 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:35.743911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:35.743972 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:35.744051 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:35.744781 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:35.745530 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:35.746783 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:35.747222 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:35.747391 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:35.752209 (+ 4818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:41343: BlockingRecv error: recv got EOF from 127.14.16.67:41343 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 07 19:38:35 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:35 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:35.932428 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:35.915589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:56733)
0807 19:38:35.916281 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:35.916305 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:35.916498 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:35.916945 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:35.916949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:35.917531 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:35.917753 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:35.917776 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:35.919248 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:35.919252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:35.920227 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:35.920238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:35.920909 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:35.920914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:35.921036 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:35.928253 (+ 7217us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:35.928277 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:35.932194 (+ 3917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":375,"threads_started":1}
W20260807 19:38:35.932715 24002 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:35.916040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:56733 (local address 127.14.16.126:44457)
0807 19:38:35.916283 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:35.916290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:35.916434 (+ 144us) server_negotiation.cc:407] Connection header received
0807 19:38:35.916650 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:35.916655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:35.916743 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:35.916879 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:35.917938 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:35.919112 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:35.920360 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:35.920776 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:35.920872 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:35.932416 (+ 11544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:56733: BlockingRecv error: recv got EOF from 127.14.16.66:56733 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 07 19:38:36 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:36 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:36.669812 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:36.660897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:44227)
0807 19:38:36.661368 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:36.661385 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:36.661487 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:36.661981 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:36.661984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:36.662389 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:36.662554 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:36.662562 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:36.663579 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:36.663582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:36.664494 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:36.664506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:36.665146 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:36.665150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:36.665286 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:36.665962 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:36.665980 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:36.669475 (+ 3495us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":238,"threads_started":1}
W20260807 19:38:36.670032 24006 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:36.660950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:44227 (local address 127.14.16.126:44457)
0807 19:38:36.661727 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:36.661731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:36.661746 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:38:36.661790 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:36.661794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:36.661849 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:36.661934 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:36.662684 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:36.663453 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:36.664617 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:36.665010 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:36.665099 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:36.669749 (+ 4650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:44227: BlockingRecv error: recv got EOF from 127.14.16.65:44227 (error 108)
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":567,"threads_started":1}
Aug 07 19:38:36 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:36 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:36.762591 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:36.753314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:54615)
0807 19:38:36.753856 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:36.753875 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:36.754034 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:36.754394 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:36.754398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:36.754933 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:36.755121 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:36.755130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:36.756532 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:36.756537 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:36.757503 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:36.757515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:36.758191 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:36.758195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:36.758316 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:36.758891 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:36.758908 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:36.762362 (+ 3454us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":296,"threads_started":1}
W20260807 19:38:36.762862 24006 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:36.753411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:54615 (local address 127.14.16.126:44457)
0807 19:38:36.753647 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:36.753655 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:36.753964 (+ 309us) server_negotiation.cc:407] Connection header received
0807 19:38:36.754164 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:36.754169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:36.754240 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:36.754355 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:36.755267 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:36.756323 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:36.757633 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:36.758058 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:36.758157 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:36.762578 (+ 4421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:54615: BlockingRecv error: recv got EOF from 127.14.16.67:54615 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 19:38:36 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:36 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:38:36.943125 24006 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:36.933576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:44389 (local address 127.14.16.126:44457)
0807 19:38:36.933834 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:36.933841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:36.934101 (+ 260us) server_negotiation.cc:407] Connection header received
0807 19:38:36.934297 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:36.934304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:36.934388 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:36.934494 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:36.935388 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:36.936233 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:36.937517 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:36.938047 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:36.938186 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:36.942908 (+ 4722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:44389: BlockingRecv error: recv got EOF from 127.14.16.66:44389 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 19:38:36.943596 24009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:36.933484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:44389)
0807 19:38:36.933996 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:36.934011 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:36.934171 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:36.934533 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:36.934538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:36.934998 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:36.935167 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:36.935176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:36.936379 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:36.936386 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:36.937379 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:36.937391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:36.938205 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:36.938211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:36.938404 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:36.939125 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:36.939143 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:36.943382 (+ 4239us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":278,"threads_started":1}
Aug 07 19:38:37 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:37 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:37.680482 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:37.670892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:57133)
0807 19:38:37.671560 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:37.671585 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:37.671718 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:37.672090 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:37.672095 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:37.672583 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:37.672781 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:37.672790 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:37.673845 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:37.673851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:37.674822 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:37.674835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:37.675662 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:37.675671 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:37.675879 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:37.676464 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:37.676481 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:37.680146 (+ 3665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":238,"threads_started":1}
W20260807 19:38:37.680685 24011 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:37.670975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:57133 (local address 127.14.16.126:44457)
0807 19:38:37.671695 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:37.671700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:37.671719 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:38:37.671862 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:37.671867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:37.671942 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:37.672057 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:37.672941 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:37.673695 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:37.674979 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:37.675492 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:37.675628 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:37.680385 (+ 4757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:57133: BlockingRecv error: recv got EOF from 127.14.16.65:57133 (error 108)
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":466,"threads_started":1}
Aug 07 19:38:37 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:37 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:37.772663 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:37.763710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:35563)
0807 19:38:37.764169 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:37.764190 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:37.764304 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:37.764572 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:37.764575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:37.765008 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:37.765187 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:37.765196 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:37.766371 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:37.766375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:37.767353 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:37.767366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:37.768156 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:37.768161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:37.768304 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:37.768915 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:37.768934 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:37.772420 (+ 3486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":260,"threads_started":1}
W20260807 19:38:37.772820 24011 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:37.763803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:35563 (local address 127.14.16.126:44457)
0807 19:38:37.764040 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:37.764047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:37.764292 (+ 245us) server_negotiation.cc:407] Connection header received
0807 19:38:37.764389 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:37.764392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:37.764452 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:37.764533 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:37.765321 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:37.766182 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:37.767529 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:37.767976 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:37.768077 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:37.772610 (+ 4533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:35563: BlockingRecv error: recv got EOF from 127.14.16.67:35563 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 19:38:37 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:37 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:37.953665 24014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:37.944709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:41411)
0807 19:38:37.945431 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:37.945448 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:37.945594 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:37.945996 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:37.946000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:37.946416 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:37.946618 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:37.946627 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:37.947715 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:37.947719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:37.948688 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:37.948699 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:37.949375 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:37.949380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:37.949509 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:37.950102 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:37.950120 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:37.953387 (+ 3267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":350,"threads_started":1}
W20260807 19:38:37.953866 24011 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:37.944806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:41411 (local address 127.14.16.126:44457)
0807 19:38:37.945045 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:37.945053 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:37.945532 (+ 479us) server_negotiation.cc:407] Connection header received
0807 19:38:37.945699 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:37.945703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:37.945782 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:37.945861 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:37.946765 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:37.947588 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:37.948824 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:37.949230 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:37.949323 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:37.953589 (+ 4266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:41411: BlockingRecv error: recv got EOF from 127.14.16.66:41411 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 19:38:38 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:38 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:38.692251 24015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:38.681576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:42037)
0807 19:38:38.682337 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:38.682354 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:38.682480 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:38.682822 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:38.682828 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:38.683341 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:38.683538 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:38.683547 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:38.684718 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:38.684733 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:38.686164 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:38.686193 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:38.687257 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:38.687262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:38.687449 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:38.688121 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:38.688140 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:38.691998 (+ 3858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":411,"threads_started":1}
W20260807 19:38:38.692517 24016 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:38.681662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:42037 (local address 127.14.16.126:44457)
0807 19:38:38.682224 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:38.682228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:38.682433 (+ 205us) server_negotiation.cc:407] Connection header received
0807 19:38:38.682580 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:38.682584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:38.682643 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:38.682735 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:38.683683 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:38.684545 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:38.686366 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:38.687036 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:38.687176 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:38.692259 (+ 5083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:42037: BlockingRecv error: recv got EOF from 127.14.16.65:42037 (error 108)
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":337,"threads_started":1}
Aug 07 19:38:38 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:38 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:38.783190 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:38.773562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:40151)
0807 19:38:38.774209 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:38.774232 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:38.774360 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:38.774690 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:38.774693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:38.775108 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:38.775345 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:38.775359 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:38.776327 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:38.776331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:38.777355 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:38.777368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:38.778080 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:38.778088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:38.778282 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:38.778985 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:38.779003 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:38.782900 (+ 3897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":322,"threads_started":1}
W20260807 19:38:38.783272 24016 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:38.773721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:40151 (local address 127.14.16.126:44457)
0807 19:38:38.773920 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:38.773925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:38.774305 (+ 380us) server_negotiation.cc:407] Connection header received
0807 19:38:38.774454 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:38.774457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:38.774518 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:38.774604 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:38.775482 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:38.776185 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:38.777482 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:38.777902 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:38.777996 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:38.783099 (+ 5103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:40151: BlockingRecv error: recv got EOF from 127.14.16.67:40151 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 19:38:38 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:38 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:38.964085 24018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:38.954629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:50379)
0807 19:38:38.955354 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:38.955372 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:38.955493 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:38.955812 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:38.955816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:38.956367 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:38.956545 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:38.956555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:38.957776 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:38.957780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:38.958693 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:38.958703 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:38.959431 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:38.959436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:38.959557 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:38.960264 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:38.960282 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:38.963790 (+ 3508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":426,"threads_started":1}
W20260807 19:38:38.964399 24016 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:38.954804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:50379 (local address 127.14.16.126:44457)
0807 19:38:38.955017 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:38.955022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:38.955456 (+ 434us) server_negotiation.cc:407] Connection header received
0807 19:38:38.955602 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:38.955606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:38.955671 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:38.955758 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:38.956690 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:38.957633 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:38.958850 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:38.959253 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:38.959377 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:38.964072 (+ 4695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:50379: BlockingRecv error: recv got EOF from 127.14.16.66:50379 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 19:38:39 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:39 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:39.702775 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:39.693570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:41783)
0807 19:38:39.694134 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:39.694151 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:39.694271 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:39.694600 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:39.694603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:39.695019 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:39.695166 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:39.695175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:39.696276 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:39.696280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:39.697200 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:39.697211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:39.697852 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:39.697856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:39.697981 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:39.698577 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:39.698593 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:39.702541 (+ 3948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":321,"threads_started":1}
W20260807 19:38:39.703058 24020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:39.693623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:41783 (local address 127.14.16.126:44457)
0807 19:38:39.694156 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:39.694160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:39.694228 (+ 68us) server_negotiation.cc:407] Connection header received
0807 19:38:39.694373 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:39.694377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:39.694437 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:39.694542 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:39.695326 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:39.696131 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:39.697326 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:39.697722 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:39.697813 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:39.702770 (+ 4957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:41783: BlockingRecv error: recv got EOF from 127.14.16.65:41783 (error 108)
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":303,"threads_started":1}
Aug 07 19:38:39 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:39 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:39.794349 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:39.784321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:44845)
0807 19:38:39.784929 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:39.784949 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:39.785069 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:39.785436 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:39.785440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:39.785987 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:39.786162 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:39.786171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:39.787173 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:39.787177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:39.788320 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:39.788333 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:39.789348 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:39.789355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:39.789540 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:39.790231 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:39.790249 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:39.794054 (+ 3805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":308,"threads_started":1}
W20260807 19:38:39.794728 24020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:39.784382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:44845 (local address 127.14.16.126:44457)
0807 19:38:39.784657 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:39.784664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:39.785036 (+ 372us) server_negotiation.cc:407] Connection header received
0807 19:38:39.785191 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:39.785194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:39.785288 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:39.785394 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:39.786302 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:39.787042 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:39.788492 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:39.789148 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:39.789282 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:39.794433 (+ 5151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:44845: BlockingRecv error: recv got EOF from 127.14.16.67:44845 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 07 19:38:39 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:39 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:39.984823 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:39.965207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:57323)
0807 19:38:39.965796 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:39.965818 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:39.965973 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:39.974710 (+ 8737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:39.974719 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:39.975471 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:39.975757 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:39.975777 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:39.977351 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:39.977359 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:39.978914 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:39.978928 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:39.979863 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:39.979869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:39.980063 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:39.980730 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:39.980749 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:39.984581 (+ 3832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":311,"threads_started":1}
W20260807 19:38:39.985155 24020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:39.965327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:57323 (local address 127.14.16.126:44457)
0807 19:38:39.965599 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:39.965603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:39.965910 (+ 307us) server_negotiation.cc:407] Connection header received
0807 19:38:39.974291 (+ 8381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:39.974299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:39.974417 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:39.974548 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:39.975951 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:39.977163 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:39.979034 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:39.979684 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:39.979818 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:39.984775 (+ 4957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:57323: BlockingRecv error: recv got EOF from 127.14.16.66:57323 (error 108)
Metrics: {"server-negotiator.queue_time_us":155}
Aug 07 19:38:40 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:40 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:40.713034 24023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:40.703740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:54745)
0807 19:38:40.704307 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:40.704331 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:40.704459 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:40.704799 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:40.704802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:40.705171 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:40.705320 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:40.705328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:40.706375 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:40.706379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:40.707541 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:40.707554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:40.708285 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:40.708291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:40.708476 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:40.709209 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:40.709228 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:40.712670 (+ 3442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":243,"threads_started":1}
W20260807 19:38:40.713193 24024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:40.703920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:54745 (local address 127.14.16.126:44457)
0807 19:38:40.704436 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:40.704440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:40.704457 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:38:40.704571 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:40.704575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:40.704635 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:40.704747 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:40.705451 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:40.706227 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:40.707684 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:40.708112 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:40.708281 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:40.712923 (+ 4642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:54745: BlockingRecv error: recv got EOF from 127.14.16.65:54745 (error 108)
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":302,"threads_started":1}
Aug 07 19:38:40 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:40 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:40.804672 24025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:40.795477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:60427)
0807 19:38:40.796031 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:40.796056 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:40.796216 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:40.796553 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:40.796556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:40.796950 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:40.797117 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:40.797125 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:40.798203 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:40.798207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:40.799170 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:40.799183 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:40.799964 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:40.799969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:40.800108 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:40.800721 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:40.800738 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:40.804412 (+ 3674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":283,"threads_started":1}
W20260807 19:38:40.805020 24024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:40.795492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:60427 (local address 127.14.16.126:44457)
0807 19:38:40.795744 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:40.795752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:40.796169 (+ 417us) server_negotiation.cc:407] Connection header received
0807 19:38:40.796344 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:40.796347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:40.796410 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:40.796491 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:40.797267 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:40.798030 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:40.799356 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:40.799768 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:40.799869 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:40.804629 (+ 4760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:60427: BlockingRecv error: recv got EOF from 127.14.16.67:60427 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 19:38:40 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:40 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:41.000656 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:40.985930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:51585)
0807 19:38:40.986479 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:40.986495 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:40.986658 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:40.987029 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:40.987034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:40.987535 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:40.987702 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:40.987711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:40.988823 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:40.988830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:40.989830 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:40.989842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:40.990624 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:40.990632 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:40.990840 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:40.996798 (+ 5958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:40.996817 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:41.000425 (+ 3608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":286,"threads_started":1}
W20260807 19:38:41.000875 24024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:40.986039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:51585 (local address 127.14.16.126:44457)
0807 19:38:40.986319 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:40.986325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:40.986590 (+ 265us) server_negotiation.cc:407] Connection header received
0807 19:38:40.986796 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:40.986804 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:40.986883 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:40.986994 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:40.987868 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:40.988673 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:40.989979 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:40.990440 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:40.990574 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:41.000622 (+ 10048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:51585: BlockingRecv error: recv got EOF from 127.14.16.66:51585 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 07 19:38:41 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:41 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:41.723568 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:41.714059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:34875)
0807 19:38:41.714574 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:41.714637 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:41.714768 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:41.715174 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:41.715177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:41.715741 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:41.716014 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:41.716030 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:41.717314 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:41.717318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:41.718249 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:41.718260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:41.719024 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:41.719028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:41.719157 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:41.719799 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:41.719817 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:41.723258 (+ 3441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":254,"threads_started":1}
W20260807 19:38:41.723848 24028 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:41.714236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:34875 (local address 127.14.16.126:44457)
0807 19:38:41.714895 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:41.714899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:41.714915 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:38:41.714958 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:41.714961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:41.715024 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:41.715128 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:41.716229 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:41.717185 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:41.718473 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:41.718885 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:41.718984 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:41.723547 (+ 4563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:34875: BlockingRecv error: recv got EOF from 127.14.16.65:34875 (error 108)
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":440,"threads_started":1}
Aug 07 19:38:41 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:41 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:41.815380 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:41.805769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:55101)
0807 19:38:41.806345 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:41.806366 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:41.806486 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:41.806823 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:41.806827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:41.807349 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:41.807524 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:41.807533 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:41.808782 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:41.808790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:41.809991 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:41.810004 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:41.810758 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:41.810764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:41.810935 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:41.811596 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:41.811613 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:41.815040 (+ 3427us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":317,"threads_started":1}
W20260807 19:38:41.815636 24028 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:41.805853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:55101 (local address 127.14.16.126:44457)
0807 19:38:41.806037 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:41.806042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:41.806446 (+ 404us) server_negotiation.cc:407] Connection header received
0807 19:38:41.806617 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:41.806621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:41.806684 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:41.806774 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:41.807688 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:41.808615 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:41.810143 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:41.810574 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:41.810721 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:41.815349 (+ 4628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:55101: BlockingRecv error: recv got EOF from 127.14.16.67:55101 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 19:38:42 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:42 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:42.012018 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:42.001603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:33531)
0807 19:38:42.002109 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:42.002126 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:42.002278 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:42.002653 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:42.002657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:42.003124 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:42.003410 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:42.003422 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:42.004812 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:42.004818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:42.006219 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:42.006240 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:42.007165 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:42.007170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:42.007335 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:42.008022 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:42.008041 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:42.011788 (+ 3747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":299,"threads_started":1}
W20260807 19:38:42.012250 24028 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:42.001774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:33531 (local address 127.14.16.126:44457)
0807 19:38:42.002018 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:42.002024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:42.002239 (+ 215us) server_negotiation.cc:407] Connection header received
0807 19:38:42.002397 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:42.002401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:42.002465 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:42.002553 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:42.003589 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:42.004620 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:42.006379 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:42.006942 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:42.007130 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:42.012006 (+ 4876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:33531: BlockingRecv error: recv got EOF from 127.14.16.66:33531 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 07 19:38:42 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:42 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:42.734551 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:42.724714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:49777)
0807 19:38:42.725565 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:42.725582 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:42.725698 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:42.726000 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:42.726003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:42.726414 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:42.726621 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:42.726630 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:42.727707 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:42.727711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:42.728658 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:42.728670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:42.729345 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:42.729349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:42.729489 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:42.730204 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:42.730223 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:42.734184 (+ 3961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":566,"threads_started":1}
W20260807 19:38:42.734936 24031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:42.724766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:49777 (local address 127.14.16.126:44457)
0807 19:38:42.725294 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:42.725298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:42.725659 (+ 361us) server_negotiation.cc:407] Connection header received
0807 19:38:42.725797 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:42.725801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:42.725861 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:42.725947 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:42.726771 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:42.727576 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:42.728797 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:42.729203 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:42.729303 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:42.734479 (+ 5176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:49777: BlockingRecv error: recv got EOF from 127.14.16.65:49777 (error 108)
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":259,"threads_started":1}
Aug 07 19:38:42 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:42 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:42.826025 24033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:42.816477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:58821)
0807 19:38:42.817080 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:42.817113 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:42.817272 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:42.817606 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:42.817610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:42.818125 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:42.818356 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:42.818367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:42.819863 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:42.819868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:42.820798 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:42.820810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:42.821458 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:42.821462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:42.821585 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:42.822190 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:42.822208 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:42.825793 (+ 3585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":316,"threads_started":1}
W20260807 19:38:42.826364 24031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:42.816605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:58821 (local address 127.14.16.126:44457)
0807 19:38:42.816845 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:42.816854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:42.817220 (+ 366us) server_negotiation.cc:407] Connection header received
0807 19:38:42.817401 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:42.817405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:42.817468 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:42.817551 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:42.818526 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:42.819705 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:42.820933 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:42.821326 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:42.821419 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:42.825996 (+ 4577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:58821: BlockingRecv error: recv got EOF from 127.14.16.67:58821 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 19:38:43 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:43 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:43.022665 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:43.013233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:57919)
0807 19:38:43.013868 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:43.013888 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:43.014049 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:43.014460 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:43.014466 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:43.014961 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:43.015133 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:43.015143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:43.016204 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:43.016210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:43.017151 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:43.017163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:43.017946 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:43.017952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:43.018134 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:43.018727 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:43.018745 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:43.022311 (+ 3566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":335,"threads_started":1}
W20260807 19:38:43.022804 24031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:43.013256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:57919 (local address 127.14.16.126:44457)
0807 19:38:43.013491 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:43.013501 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:43.014010 (+ 509us) server_negotiation.cc:407] Connection header received
0807 19:38:43.014172 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:43.014180 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:43.014287 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:43.014404 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:43.015318 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:43.016051 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:43.017309 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:43.017749 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:43.017901 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:43.022530 (+ 4629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:57919: BlockingRecv error: recv got EOF from 127.14.16.66:57919 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 19:38:43 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:43 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:43.745587 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:43.735691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:46873)
0807 19:38:43.736293 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:43.736328 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:43.736473 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:43.736883 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:43.736890 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:43.737409 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:43.737563 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:43.737571 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:43.738844 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:43.738852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:43.739815 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:43.739827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:43.740671 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:43.740676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:43.740874 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:43.741527 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:43.741551 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:43.745349 (+ 3798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":260,"threads_started":1}
W20260807 19:38:43.745908 24036 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:43.735815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:46873 (local address 127.14.16.126:44457)
0807 19:38:43.736537 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:43.736544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:43.736562 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:38:43.736651 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:43.736655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:43.736711 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:43.736846 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:43.737866 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:43.738677 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:43.739973 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:43.740504 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:43.740655 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:43.745589 (+ 4934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:46873: BlockingRecv error: recv got EOF from 127.14.16.65:46873 (error 108)
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":423,"threads_started":1}
Aug 07 19:38:43 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:43 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:43.835840 24037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:43.827137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:54991)
0807 19:38:43.827708 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:43.827728 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:43.827862 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:43.828134 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:43.828137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:43.828577 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:43.828755 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:43.828773 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:43.829718 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:43.829721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:43.830636 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:43.830647 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:43.831396 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:43.831402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:43.831533 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:43.832209 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:43.832227 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:43.835593 (+ 3366us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":327,"threads_started":1}
W20260807 19:38:43.836108 24036 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:43.827180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:54991 (local address 127.14.16.126:44457)
0807 19:38:43.827434 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:43.827443 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:43.827827 (+ 384us) server_negotiation.cc:407] Connection header received
0807 19:38:43.827962 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:43.827965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:43.828028 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:43.828110 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:43.828891 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:43.829600 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:43.830783 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:43.831177 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:43.831269 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:43.835865 (+ 4596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:54991: BlockingRecv error: recv got EOF from 127.14.16.67:54991 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 19:38:44 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:44 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:44.033187 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:44.023831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:32791)
0807 19:38:44.024368 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:44.024385 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:44.024501 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:44.024818 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:44.024821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:44.025227 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:44.025386 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:44.025395 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:44.026546 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:44.026552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:44.027755 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:44.027770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:44.028750 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:44.028754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:44.028881 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:44.029437 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:44.029456 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:44.032926 (+ 3470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":295,"threads_started":1}
W20260807 19:38:44.033387 24036 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:44.024001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:32791 (local address 127.14.16.126:44457)
0807 19:38:44.024194 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:44.024199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:44.024461 (+ 262us) server_negotiation.cc:407] Connection header received
0807 19:38:44.024599 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:44.024603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:44.024683 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:44.024771 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:44.025605 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:44.026358 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:44.027909 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:44.028471 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:44.028610 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:44.033103 (+ 4493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:32791: BlockingRecv error: recv got EOF from 127.14.16.66:32791 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 19:38:44 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:44 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.65@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:44.756759 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:44.746951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.65:57347)
0807 19:38:44.747527 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:44.747547 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:44.747653 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:44.748001 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:44.748005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:44.748428 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:44.748627 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:44.748636 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:44.749691 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:44.749696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:44.750863 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:44.750878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:44.751672 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:44.751676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:44.751821 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:44.752472 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:44.752491 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:44.756394 (+ 3903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":287,"threads_started":1}
W20260807 19:38:44.756925 24040 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:44.747190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:57347 (local address 127.14.16.126:44457)
0807 19:38:44.747734 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:44.747739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:44.747754 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:38:44.747799 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:44.747802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:44.747865 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:44.747956 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:44.748760 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:44.749538 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:44.751002 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:44.751526 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:44.751624 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:44.756654 (+ 5030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.65:57347: BlockingRecv error: recv got EOF from 127.14.16.65:57347 (error 108)
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":292,"threads_started":1}
Aug 07 19:38:44 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:44 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.67@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:44.846349 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:44.836851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.67:57825)
0807 19:38:44.837408 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:44.837431 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:44.837581 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:44.837880 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:44.837884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:44.838346 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:44.838498 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:44.838507 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:44.839667 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:44.839671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:44.840621 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:44.840634 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:44.841353 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:44.841358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:44.841493 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:44.842115 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:44.842134 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:44.846006 (+ 3872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":321,"threads_started":1}
W20260807 19:38:44.846452 24040 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:44.836979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:57825 (local address 127.14.16.126:44457)
0807 19:38:44.837233 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:44.837237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:44.837504 (+ 267us) server_negotiation.cc:407] Connection header received
0807 19:38:44.837682 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:44.837686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:44.837751 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:44.837834 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:44.838698 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:44.839468 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:44.840761 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:44.841189 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:44.841285 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:44.846211 (+ 4926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.67:57825: BlockingRecv error: recv got EOF from 127.14.16.67:57825 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 07 19:38:45 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:38:45 dist-test-slave-07c2 krb5kdc[22938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.16.66@KRBTEST.COM for oryx/127.14.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:38:45.044412 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:45.034376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44457 (local address 127.14.16.66:51471)
0807 19:38:45.034867 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:45.034884 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:45.034998 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:45.035364 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:45.035368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:45.035866 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:45.036108 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:45.036119 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:45.037399 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:45.037404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:45.038838 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:45.038851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:45.039645 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:45.039651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:45.039858 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:45.040446 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:45.040464 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:45.044133 (+ 3669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.16.126:44457: Server oryx/127.14.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":244,"threads_started":1}
W20260807 19:38:45.044587 24040 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:45.034526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:51471 (local address 127.14.16.126:44457)
0807 19:38:45.034761 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:45.034767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:45.034962 (+ 195us) server_negotiation.cc:407] Connection header received
0807 19:38:45.035097 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:45.035101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:45.035177 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:45.035272 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:45.036257 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:45.037265 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:45.038977 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:45.039491 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:45.039629 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:45.044355 (+ 4726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.16.66:51471: BlockingRecv error: recv got EOF from 127.14.16.66:51471 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:38:45.543521 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 23548
I20260807 19:38:45.556357 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:45.544726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49174 (local address 127.14.16.65:37035)
0807 19:38:45.545644 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:45.545652 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:45.545677 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:38:45.545750 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:45.545756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:45.546011 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:45.546181 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:45.547686 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:45.548721 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:45.549917 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:45.550506 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:45.550611 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:45.553192 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:45.553201 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:45.553216 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:45.553266 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:45.555868 (+ 2602us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.16.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 19:38:45.556145 (+ 277us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49174: Request ticket server kudu/127.14.16.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":597,"threads_started":1}
W20260807 19:38:45.556484 24044 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49174: Request ticket server kudu/127.14.16.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 19:38:45.556540 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 23548: Not authorized: Client connection negotiation failed: client connection to 127.14.16.65:37035: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.16.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 19:38:45.556810 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 23548
I20260807 19:38:45.579712 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 23703
I20260807 19:38:45.593194 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:45.580914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40338 (local address 127.14.16.66:37769)
0807 19:38:45.581575 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:45.581582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:45.581608 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:38:45.581682 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:45.581688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:45.581929 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:45.582149 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:45.583607 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:45.584697 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:45.586049 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:45.586624 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:45.586808 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:45.589533 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:45.589542 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:45.589556 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:45.589593 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:45.592447 (+ 2854us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.16.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 19:38:45.592832 (+ 385us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40338: Request ticket server kudu/127.14.16.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":355,"threads_started":1}
W20260807 19:38:45.593243 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 23703: Not authorized: Client connection negotiation failed: client connection to 127.14.16.66:37769: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.16.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 19:38:45.593324 24045 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40338: Request ticket server kudu/127.14.16.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 19:38:45.593410 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 23703
I20260807 19:38:45.613387 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 23843
I20260807 19:38:45.626317 24046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:45.614645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33854 (local address 127.14.16.67:35675)
0807 19:38:45.615351 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:45.615358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:45.615387 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:38:45.615457 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:45.615465 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:45.615715 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:45.615925 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:45.617322 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:45.618364 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:45.619746 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:45.620304 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:45.620466 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:45.623017 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:45.623026 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:45.623039 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:45.623080 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:45.625602 (+ 2522us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.16.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 19:38:45.625961 (+ 359us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33854: Request ticket server kudu/127.14.16.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":338,"threads_started":1}
W20260807 19:38:45.626314 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 23843: Not authorized: Client connection negotiation failed: client connection to 127.14.16.67:35675: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.16.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 19:38:45.626436 24046 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33854: Request ticket server kudu/127.14.16.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 19:38:45.626494 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 23843
I20260807 19:38:45.643682 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 23477
I20260807 19:38:45.645068 23541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:45.765967 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 23477
2026-08-07T19:38:45Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21802 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T19:38:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:38:45Z Disabled control of system clock
I20260807 19:38:45.856194 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:44225
--webserver_interface=127.14.16.126
--webserver_port=0
--builtin_ntp_servers=127.14.16.84:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:44225
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 19:38:46.198789 24051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:46.199091 24051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:46.199229 24051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:46.206562 24051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:38:46.206676 24051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:46.206774 24051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:46.206871 24051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:38:46.206948 24051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:38:46.216300 24051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:44225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:44225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:46.217685 24051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:46.219555 24051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:46.231676 24057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:46.231717 24060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:46.231637 24056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:46.232462 24051 server_base.cc:1034] running on GCE node
I20260807 19:38:46.233214 24051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:46.234539 24051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:46.235725 24051 hybrid_clock.cc:648] HybridClock initialized: now 1786131526235676 us; error 55 us; skew 500 ppm
I20260807 19:38:46.236075 24051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:38:46.248715 24051 webserver.cc:460] Webserver started at http://127.14.16.126:35543/ using document root <none> and password file <none>
I20260807 19:38:46.249492 24051 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:46.249585 24051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:46.249934 24051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:46.252945 24051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "b9aaf259df684d3d8d36270e47681175"
format_stamp: "Formatted at 2026-08-07 19:38:46 on dist-test-slave-07c2"
server_key: "b86abb6566ea0fe5f0c5ac78ccbcc3b8"
server_key_iv: "df60da167d9879a10ab8d1f581d40289"
server_key_version: "encryptionkey@0"
I20260807 19:38:46.253746 24051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "b9aaf259df684d3d8d36270e47681175"
format_stamp: "Formatted at 2026-08-07 19:38:46 on dist-test-slave-07c2"
server_key: "b86abb6566ea0fe5f0c5ac78ccbcc3b8"
server_key_iv: "df60da167d9879a10ab8d1f581d40289"
server_key_version: "encryptionkey@0"
I20260807 19:38:46.258985 24051 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:38:46.262526 24066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:46.263959 24051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 19:38:46.264158 24051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "b9aaf259df684d3d8d36270e47681175"
format_stamp: "Formatted at 2026-08-07 19:38:46 on dist-test-slave-07c2"
server_key: "b86abb6566ea0fe5f0c5ac78ccbcc3b8"
server_key_iv: "df60da167d9879a10ab8d1f581d40289"
server_key_version: "encryptionkey@0"
I20260807 19:38:46.264389 24051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:46.283656 24051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:46.284497 24051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:46.284801 24051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:46.310510 24051 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:44225
I20260807 19:38:46.310626 24120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:44225 every 8 connection(s)
I20260807 19:38:46.312021 24051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:38:46.315930 24121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:46.322086 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 24051
I20260807 19:38:46.322270 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:38:46.322965 14401 external_mini_cluster.cc:1428] Setting key 9240914f4cc025cfdaef8652e696e992
I20260807 19:38:46.326148 24121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175: Bootstrap starting.
I20260807 19:38:46.329483 24121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:46.330500 24121 log.cc:824] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:46.333525 24121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175: No bootstrap required, opened a new log
I20260807 19:38:46.336719 24124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:46.324995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37862 (local address 127.14.16.126:44225)
0807 19:38:46.326036 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:46.326046 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:46.326088 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:38:46.327864 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:46.327885 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:46.328303 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:46.328619 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:46.329276 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:46.330680 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:46.332374 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:46.333042 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:46.333188 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:46.334401 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:46.334408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:46.334420 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:46.334935 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:46.335012 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:46.335034 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:46.335133 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:46.335420 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 19:38:46.335784 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":376,"threads_started":1}
I20260807 19:38:46.342046 24121 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9aaf259df684d3d8d36270e47681175" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44225 } }
I20260807 19:38:46.342671 24121 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:46.342796 24121 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9aaf259df684d3d8d36270e47681175, State: Initialized, Role: FOLLOWER
I20260807 19:38:46.343442 24121 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9aaf259df684d3d8d36270e47681175" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44225 } }
I20260807 19:38:46.343732 24121 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:38:46.343885 24121 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:38:46.344075 24121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:46.346343 24121 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9aaf259df684d3d8d36270e47681175" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44225 } }
I20260807 19:38:46.346837 24121 leader_election.cc:302] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9aaf259df684d3d8d36270e47681175; no voters:
I20260807 19:38:46.347482 24121 leader_election.cc:288] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:38:46.347633 24126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:46.348878 24126 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [term 1 LEADER]: Becoming Leader. State: Replica: b9aaf259df684d3d8d36270e47681175, State: Running, Role: LEADER
I20260807 19:38:46.349442 24126 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9aaf259df684d3d8d36270e47681175" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44225 } }
I20260807 19:38:46.349588 24121 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:38:46.352589 24128 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9aaf259df684d3d8d36270e47681175. Latest consensus state: current_term: 1 leader_uuid: "b9aaf259df684d3d8d36270e47681175" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9aaf259df684d3d8d36270e47681175" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44225 } } }
I20260807 19:38:46.352566 24127 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9aaf259df684d3d8d36270e47681175" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9aaf259df684d3d8d36270e47681175" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44225 } } }
I20260807 19:38:46.353024 24128 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:46.353062 24127 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:46.360502 24133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:38:46.365733 24133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:38:46.374603 24133 catalog_manager.cc:1347] Generated new cluster ID: 31dbebfad55b48a6aec0d7d961383eef
I20260807 19:38:46.374779 24133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:38:46.388271 24133 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:38:46.389523 24133 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:38:46.395928 24133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9aaf259df684d3d8d36270e47681175: Generated new TSK 0
I20260807 19:38:46.396727 24133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:38:46.412729 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44225
--builtin_ntp_servers=127.14.16.84:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 19:38:46.762012 24145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:46.762380 24145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:46.762552 24145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:46.770426 24145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:46.770571 24145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:46.770759 24145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:38:46.780592 24145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44225
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:46.781991 24145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:46.783962 24145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:46.798345 24150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:46.798247 24151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:46.798456 24153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:46.799211 24145 server_base.cc:1034] running on GCE node
I20260807 19:38:46.799898 24145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:46.800935 24145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:46.802179 24145 hybrid_clock.cc:648] HybridClock initialized: now 1786131526802103 us; error 56 us; skew 500 ppm
I20260807 19:38:46.802557 24145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:38:46.804955 24145 webserver.cc:460] Webserver started at http://127.14.16.65:35623/ using document root <none> and password file <none>
I20260807 19:38:46.805640 24145 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:46.805737 24145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:46.806015 24145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:46.809031 24145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "45f4bfea72aa4ae383716233742a1db3"
format_stamp: "Formatted at 2026-08-07 19:38:46 on dist-test-slave-07c2"
server_key: "fdc0b6370115542b02036dbd408a615c"
server_key_iv: "b81edf8d2168ccaa06cf6e1cc81c3a47"
server_key_version: "encryptionkey@0"
I20260807 19:38:46.809808 24145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "45f4bfea72aa4ae383716233742a1db3"
format_stamp: "Formatted at 2026-08-07 19:38:46 on dist-test-slave-07c2"
server_key: "fdc0b6370115542b02036dbd408a615c"
server_key_iv: "b81edf8d2168ccaa06cf6e1cc81c3a47"
server_key_version: "encryptionkey@0"
I20260807 19:38:46.815198 24145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:38:46.818753 24160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:46.820767 24145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 19:38:46.820916 24145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "45f4bfea72aa4ae383716233742a1db3"
format_stamp: "Formatted at 2026-08-07 19:38:46 on dist-test-slave-07c2"
server_key: "fdc0b6370115542b02036dbd408a615c"
server_key_iv: "b81edf8d2168ccaa06cf6e1cc81c3a47"
server_key_version: "encryptionkey@0"
I20260807 19:38:46.821113 24145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:46.864333 24145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:46.865177 24145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:46.865442 24145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:46.866400 24145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:46.868033 24145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:46.868122 24145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:46.868284 24145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:46.868363 24145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:46.903364 24145 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:38797
I20260807 19:38:46.903457 24277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:38797 every 8 connection(s)
I20260807 19:38:46.904937 24145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:38:46.912804 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 24145
I20260807 19:38:46.913017 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:38:46.913697 14401 external_mini_cluster.cc:1428] Setting key d7ea9c1d2b3f7e01282947976aa04b76
I20260807 19:38:46.916769 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44225
--builtin_ntp_servers=127.14.16.84:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 19:38:46.917737 24280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:46.907543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:34241 (local address 127.14.16.126:44225)
0807 19:38:46.908030 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:46.908034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:46.909087 (+ 1053us) server_negotiation.cc:407] Connection header received
0807 19:38:46.910869 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:46.910874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:46.910971 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:46.911093 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:46.912193 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:46.912989 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:46.914637 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:46.915208 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:46.915385 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:46.916968 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:46.916976 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:46.916983 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:46.917005 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:46.917055 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:46.917068 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:46.917138 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:46.917407 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 19:38:46.917485 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":267,"threads_started":1}
I20260807 19:38:46.919349 24281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:46.908021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44225 (local address 127.14.16.65:34241)
0807 19:38:46.908924 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:46.908966 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:46.910593 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:46.911372 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:46.911381 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:46.911472 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:46.912024 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:46.912041 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:46.913187 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:46.913191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:46.914473 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:46.914485 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:46.915499 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:46.915510 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:46.915794 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:46.916536 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:38:46.916541 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:46.916596 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:38:46.916710 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:46.917183 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:46.917197 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:46.917207 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 19:38:46.917392 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 19:38:46.917851 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":271,"threads_started":1}
I20260807 19:38:46.920848 24278 heartbeater.cc:347] Connected to a master server at 127.14.16.126:44225
I20260807 19:38:46.921155 24278 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:46.921916 24278 heartbeater.cc:511] Master 127.14.16.126:44225 requested a full tablet report, sending...
I20260807 19:38:46.923995 24083 ts_manager.cc:194] Registered new tserver with Master: 45f4bfea72aa4ae383716233742a1db3 (127.14.16.65:38797)
I20260807 19:38:46.925587 24083 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.16.65:34241
W20260807 19:38:47.258322 24282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:47.258613 24282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:47.258764 24282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:47.266990 24282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:47.267093 24282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:47.267264 24282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:38:47.276800 24282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44225
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:47.278102 24282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:47.279949 24282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:47.293637 24287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:47.293689 24290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:47.293597 24288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:47.294644 24282 server_base.cc:1034] running on GCE node
I20260807 19:38:47.295361 24282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:47.296361 24282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:47.297602 24282 hybrid_clock.cc:648] HybridClock initialized: now 1786131527297556 us; error 43 us; skew 500 ppm
I20260807 19:38:47.297945 24282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:38:47.300447 24282 webserver.cc:460] Webserver started at http://127.14.16.66:33539/ using document root <none> and password file <none>
I20260807 19:38:47.301182 24282 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:47.301330 24282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:47.301667 24282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:47.304661 24282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "6d14f36848ed491ca6ab51f8adb53e4e"
format_stamp: "Formatted at 2026-08-07 19:38:47 on dist-test-slave-07c2"
server_key: "f315719081ecec645ddba54c84a047c5"
server_key_iv: "e53098c6a335b8b042c59f5d1980630e"
server_key_version: "encryptionkey@0"
I20260807 19:38:47.305476 24282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "6d14f36848ed491ca6ab51f8adb53e4e"
format_stamp: "Formatted at 2026-08-07 19:38:47 on dist-test-slave-07c2"
server_key: "f315719081ecec645ddba54c84a047c5"
server_key_iv: "e53098c6a335b8b042c59f5d1980630e"
server_key_version: "encryptionkey@0"
I20260807 19:38:47.310600 24282 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:38:47.314258 24297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:47.315757 24282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:38:47.315948 24282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "6d14f36848ed491ca6ab51f8adb53e4e"
format_stamp: "Formatted at 2026-08-07 19:38:47 on dist-test-slave-07c2"
server_key: "f315719081ecec645ddba54c84a047c5"
server_key_iv: "e53098c6a335b8b042c59f5d1980630e"
server_key_version: "encryptionkey@0"
I20260807 19:38:47.316217 24282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:47.338274 24282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:47.339154 24282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:47.339516 24282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:47.340492 24282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:47.341989 24282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:47.342114 24282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:47.342254 24282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:47.342350 24282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:47.376807 24282 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:44363
I20260807 19:38:47.376901 24411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:44363 every 8 connection(s)
I20260807 19:38:47.378199 24282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:38:47.386758 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 24282
I20260807 19:38:47.386936 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:38:47.387523 14401 external_mini_cluster.cc:1428] Setting key d93f5bbaabc6c64e77f18f66ae8a6def
I20260807 19:38:47.390350 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44225
--builtin_ntp_servers=127.14.16.84:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 19:38:47.392194 24280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:47.380981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:46259 (local address 127.14.16.126:44225)
0807 19:38:47.381197 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:47.381203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:47.382568 (+ 1365us) server_negotiation.cc:407] Connection header received
0807 19:38:47.384767 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:47.384773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:47.384873 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:47.384989 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:47.386476 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:47.387659 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:47.389263 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:47.389687 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:47.389819 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:47.391326 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:47.391333 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:47.391337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:47.391359 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:47.391401 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:47.391414 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:47.391511 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:47.391781 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 19:38:47.391881 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:38:47.393219 24414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:47.381487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44225 (local address 127.14.16.66:46259)
0807 19:38:47.382393 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:47.382440 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:47.384431 (+ 1991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:47.385236 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:47.385244 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:47.385331 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:47.386178 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:47.386204 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:47.387811 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:47.387815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:47.389102 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:47.389113 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:47.389922 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:47.389931 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:47.390173 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:47.390936 (+ 763us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:38:47.390938 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:47.390989 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:38:47.391084 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:47.391512 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:47.391525 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:47.391536 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:38:47.391774 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 19:38:47.392180 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":289,"threads_started":1}
I20260807 19:38:47.394686 24412 heartbeater.cc:347] Connected to a master server at 127.14.16.126:44225
I20260807 19:38:47.394995 24412 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:47.395788 24412 heartbeater.cc:511] Master 127.14.16.126:44225 requested a full tablet report, sending...
I20260807 19:38:47.397320 24083 ts_manager.cc:194] Registered new tserver with Master: 6d14f36848ed491ca6ab51f8adb53e4e (127.14.16.66:44363)
I20260807 19:38:47.398518 24083 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.16.66:46259
W20260807 19:38:47.737820 24415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:47.738106 24415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:47.738190 24415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:47.745311 24415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:47.745415 24415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:47.745519 24415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:38:47.754493 24415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44225
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:47.755709 24415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:47.757395 24415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:47.770308 24420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:47.770346 24421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:47.770493 24423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:47.771136 24415 server_base.cc:1034] running on GCE node
I20260807 19:38:47.771745 24415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:47.772661 24415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:47.773890 24415 hybrid_clock.cc:648] HybridClock initialized: now 1786131527773838 us; error 63 us; skew 500 ppm
I20260807 19:38:47.774197 24415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:38:47.776511 24415 webserver.cc:460] Webserver started at http://127.14.16.67:43685/ using document root <none> and password file <none>
I20260807 19:38:47.777226 24415 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:47.777360 24415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:47.777691 24415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:47.780522 24415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "205116e6a9284c92ab1575911cfa52ea"
format_stamp: "Formatted at 2026-08-07 19:38:47 on dist-test-slave-07c2"
server_key: "e32d5f82b81877ad658401c3162a93a5"
server_key_iv: "a32dd30550281853ec2fc74a140178f7"
server_key_version: "encryptionkey@0"
I20260807 19:38:47.781318 24415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "205116e6a9284c92ab1575911cfa52ea"
format_stamp: "Formatted at 2026-08-07 19:38:47 on dist-test-slave-07c2"
server_key: "e32d5f82b81877ad658401c3162a93a5"
server_key_iv: "a32dd30550281853ec2fc74a140178f7"
server_key_version: "encryptionkey@0"
I20260807 19:38:47.786203 24415 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:38:47.789549 24430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:47.790825 24415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:38:47.791002 24415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "205116e6a9284c92ab1575911cfa52ea"
format_stamp: "Formatted at 2026-08-07 19:38:47 on dist-test-slave-07c2"
server_key: "e32d5f82b81877ad658401c3162a93a5"
server_key_iv: "a32dd30550281853ec2fc74a140178f7"
server_key_version: "encryptionkey@0"
I20260807 19:38:47.791214 24415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:47.819722 24415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:47.820662 24415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:47.820963 24415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:47.821967 24415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:47.823506 24415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:47.823640 24415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:47.823765 24415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:47.823856 24415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:47.858309 24415 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:35353
I20260807 19:38:47.858368 24545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:35353 every 8 connection(s)
I20260807 19:38:47.859660 24415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:38:47.869731 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 24415
I20260807 19:38:47.869911 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:38:47.870484 14401 external_mini_cluster.cc:1428] Setting key c90775a892325d874fae2be93c00b98f
I20260807 19:38:47.873306 24280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:47.862679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:50801 (local address 127.14.16.126:44225)
0807 19:38:47.862897 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:47.862902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:47.864159 (+ 1257us) server_negotiation.cc:407] Connection header received
0807 19:38:47.866194 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:47.866198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:47.866269 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:47.866362 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:47.867462 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:47.868324 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:47.870379 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:47.870830 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:47.870921 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:47.872435 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:47.872440 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:47.872443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:47.872461 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:47.872497 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:47.872509 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:47.872579 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:47.872844 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 19:38:47.872941 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:38:47.874154 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:47.863036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44225 (local address 127.14.16.67:50801)
0807 19:38:47.863993 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:47.864030 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:47.865857 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:47.866555 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:47.866563 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:47.866674 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:47.867217 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:47.867235 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:47.868463 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:47.868466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:47.870244 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:47.870260 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:47.871050 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:47.871056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:47.871292 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:47.872118 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:38:47.872120 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:47.872165 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:38:47.872240 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:47.872612 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:47.872626 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:47.872636 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 19:38:47.872795 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 19:38:47.873071 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":330,"threads_started":1}
I20260807 19:38:47.875630 24546 heartbeater.cc:347] Connected to a master server at 127.14.16.126:44225
I20260807 19:38:47.875932 24546 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:47.876670 24546 heartbeater.cc:511] Master 127.14.16.126:44225 requested a full tablet report, sending...
I20260807 19:38:47.878067 24083 ts_manager.cc:194] Registered new tserver with Master: 205116e6a9284c92ab1575911cfa52ea (127.14.16.67:35353)
I20260807 19:38:47.878829 24083 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.16.67:50801
I20260807 19:38:47.886281 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 19:38:47.892963 24280 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:47.891254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37878 (local address 127.14.16.126:44225)
0807 19:38:47.891509 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:47.891513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:47.892008 (+ 495us) server_negotiation.cc:407] Connection header received
0807 19:38:47.892155 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:47.892159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:47.892227 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:47.892321 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:47.892776 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37878: BlockingRecv error: recv got EOF from 127.0.0.1:37878 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 19:38:47.893446 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 24145
I20260807 19:38:47.901599 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:47.894528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47252 (local address 127.14.16.65:38797)
0807 19:38:47.895126 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:47.895132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:47.895159 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:38:47.895215 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:47.895221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:47.895504 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:47.895629 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:47.896084 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:47.897089 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:47.898304 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:47.898880 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:47.898978 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:47.900091 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:47.900099 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:47.900113 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:47.900719 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:47.900793 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:47.900814 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:47.900912 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:47.901053 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 19:38:47.901187 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":328,"threads_started":1}
I20260807 19:38:47.903054 24273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:47.986537 24278 heartbeater.cc:503] Master 127.14.16.126:44225 was elected leader, sending a full tablet report...
I20260807 19:38:47.988312 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 24145
I20260807 19:38:48.007994 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 24282
I20260807 19:38:48.016283 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:48.008931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44854 (local address 127.14.16.66:44363)
0807 19:38:48.009498 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:48.009505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:48.009534 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:38:48.009608 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:48.009614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:48.009834 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:48.009975 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:48.010319 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:48.011376 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:48.012616 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:48.013216 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:48.013339 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:48.014478 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:48.014486 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:48.014499 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:48.015172 (+ 673us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:48.015238 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:48.015260 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:48.015652 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:48.015789 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 19:38:48.015928 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":312,"threads_started":1}
I20260807 19:38:48.017572 24407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:48.102277 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 24282
I20260807 19:38:48.121058 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 24415
I20260807 19:38:48.129696 24560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:48.122369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.14.16.67:35353)
0807 19:38:48.123098 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:48.123113 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:48.123144 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:38:48.123204 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:48.123210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:48.123523 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:48.123654 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:48.124070 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:48.125109 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:48.126400 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:48.126965 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:48.127064 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:48.128281 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:48.128291 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:48.128305 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:48.128924 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:48.128970 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:48.128989 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:48.129094 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:48.129262 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 19:38:48.129382 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":430,"threads_started":1}
I20260807 19:38:48.131099 24541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:48.214411 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 24415
I20260807 19:38:48.232981 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 24051
I20260807 19:38:48.234232 24116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:48.338943 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 24051
2026-08-07T19:38:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2566 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T19:38:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:38:48Z Disabled control of system clock
I20260807 19:38:48.425037 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:46243
--webserver_interface=127.14.16.126
--webserver_port=0
--builtin_ntp_servers=127.14.16.84:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:46243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 19:38:48.761003 24566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:48.761305 24566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:48.761487 24566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:48.768844 24566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:38:48.768958 24566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:48.769048 24566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:48.769161 24566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:38:48.769256 24566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:38:48.778782 24566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:46243
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:46243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:48.780210 24566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:48.781991 24566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:48.793991 24574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:48.793946 24571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:48.794049 24572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:48.800457 24566 server_base.cc:1034] running on GCE node
I20260807 19:38:48.801401 24566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:48.802917 24566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:48.804147 24566 hybrid_clock.cc:648] HybridClock initialized: now 1786131528804103 us; error 41 us; skew 500 ppm
I20260807 19:38:48.804507 24566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:38:48.806806 24566 webserver.cc:460] Webserver started at http://127.14.16.126:37777/ using document root <none> and password file <none>
I20260807 19:38:48.807595 24566 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:48.807727 24566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:48.808070 24566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:48.814985 24566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "9c96bcdd76e640a283481611fcbd15a9"
format_stamp: "Formatted at 2026-08-07 19:38:48 on dist-test-slave-07c2"
server_key: "b4cb217acb1699bef99189d908d1abca"
server_key_iv: "71d5ba9f85101d65af81856f8e8aa6f1"
server_key_version: "encryptionkey@0"
I20260807 19:38:48.815919 24566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "9c96bcdd76e640a283481611fcbd15a9"
format_stamp: "Formatted at 2026-08-07 19:38:48 on dist-test-slave-07c2"
server_key: "b4cb217acb1699bef99189d908d1abca"
server_key_iv: "71d5ba9f85101d65af81856f8e8aa6f1"
server_key_version: "encryptionkey@0"
I20260807 19:38:48.821437 24566 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:38:48.824925 24580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:48.826261 24566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:38:48.826455 24566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "9c96bcdd76e640a283481611fcbd15a9"
format_stamp: "Formatted at 2026-08-07 19:38:48 on dist-test-slave-07c2"
server_key: "b4cb217acb1699bef99189d908d1abca"
server_key_iv: "71d5ba9f85101d65af81856f8e8aa6f1"
server_key_version: "encryptionkey@0"
I20260807 19:38:48.826687 24566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:48.832489 24566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:48.833282 24566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:48.833591 24566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:48.852489 24566 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:46243
I20260807 19:38:48.852530 24633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:46243 every 8 connection(s)
I20260807 19:38:48.853961 24566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:38:48.857767 24634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:48.863663 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 24566
I20260807 19:38:48.863904 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:38:48.864779 14401 external_mini_cluster.cc:1428] Setting key 9ee10b50e13cb394d3bba3f322fb81e0
I20260807 19:38:48.869676 24634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9: Bootstrap starting.
I20260807 19:38:48.873440 24634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:48.873764 24637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:48.867069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58872 (local address 127.14.16.126:46243)
0807 19:38:48.868247 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:48.868259 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:48.868303 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:38:48.870073 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:48.870097 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:48.870517 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:48.870842 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:48.872081 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:48.872087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:48.872100 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:48.872114 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:48.872173 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:48.872192 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:48.872370 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:48.872523 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 19:38:48.872819 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":503,"threads_started":1}
I20260807 19:38:48.874595 24634 log.cc:824] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:48.877463 24634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9: No bootstrap required, opened a new log
I20260807 19:38:48.884584 24634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c96bcdd76e640a283481611fcbd15a9" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46243 } }
I20260807 19:38:48.884984 24634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:48.885090 24634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c96bcdd76e640a283481611fcbd15a9, State: Initialized, Role: FOLLOWER
I20260807 19:38:48.886003 24634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c96bcdd76e640a283481611fcbd15a9" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46243 } }
I20260807 19:38:48.886369 24634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:38:48.886559 24634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:38:48.886821 24634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:48.889496 24634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c96bcdd76e640a283481611fcbd15a9" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46243 } }
I20260807 19:38:48.890002 24634 leader_election.cc:302] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c96bcdd76e640a283481611fcbd15a9; no voters:
I20260807 19:38:48.890604 24634 leader_election.cc:288] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:38:48.890794 24639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:48.891705 24639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [term 1 LEADER]: Becoming Leader. State: Replica: 9c96bcdd76e640a283481611fcbd15a9, State: Running, Role: LEADER
I20260807 19:38:48.892382 24639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c96bcdd76e640a283481611fcbd15a9" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46243 } }
I20260807 19:38:48.893177 24634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:38:48.895223 24641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c96bcdd76e640a283481611fcbd15a9. Latest consensus state: current_term: 1 leader_uuid: "9c96bcdd76e640a283481611fcbd15a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c96bcdd76e640a283481611fcbd15a9" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46243 } } }
I20260807 19:38:48.895627 24641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:48.896900 24640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c96bcdd76e640a283481611fcbd15a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c96bcdd76e640a283481611fcbd15a9" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46243 } } }
I20260807 19:38:48.897684 24640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:48.902694 24646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:38:48.906994 24646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:38:48.915627 24646 catalog_manager.cc:1347] Generated new cluster ID: ddd72c84a93948f9bde273cd7f54dfd3
I20260807 19:38:48.915741 24646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:38:48.936278 24646 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:38:48.936713 24646 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:38:48.946837 24646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c96bcdd76e640a283481611fcbd15a9: Generated new TSK 0
I20260807 19:38:48.947667 24646 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:38:48.955137 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:46243
--builtin_ntp_servers=127.14.16.84:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 19:38:49.293385 24660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:49.293745 24660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:49.293917 24660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:49.301608 24660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:49.301728 24660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:49.301937 24660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:38:49.311533 24660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:46243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:49.312912 24660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:49.314795 24660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:49.328831 24666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:49.328922 24665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:49.328910 24668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:49.332834 24660 server_base.cc:1034] running on GCE node
I20260807 19:38:49.333537 24660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:49.334610 24660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:49.335829 24660 hybrid_clock.cc:648] HybridClock initialized: now 1786131529335779 us; error 42 us; skew 500 ppm
I20260807 19:38:49.336177 24660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:38:49.338574 24660 webserver.cc:460] Webserver started at http://127.14.16.65:36811/ using document root <none> and password file <none>
I20260807 19:38:49.339370 24660 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:49.339498 24660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:49.339854 24660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:49.342906 24660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "b7295d2ce42840fbbd4116475cf8b462"
format_stamp: "Formatted at 2026-08-07 19:38:49 on dist-test-slave-07c2"
server_key: "6764a1931ab65bf8278b98a75c505eb6"
server_key_iv: "767e715f255430c8533d815ff460ac5a"
server_key_version: "encryptionkey@0"
I20260807 19:38:49.343760 24660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "b7295d2ce42840fbbd4116475cf8b462"
format_stamp: "Formatted at 2026-08-07 19:38:49 on dist-test-slave-07c2"
server_key: "6764a1931ab65bf8278b98a75c505eb6"
server_key_iv: "767e715f255430c8533d815ff460ac5a"
server_key_version: "encryptionkey@0"
I20260807 19:38:49.349080 24660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 19:38:49.352828 24674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:49.354226 24660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 19:38:49.354398 24660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "b7295d2ce42840fbbd4116475cf8b462"
format_stamp: "Formatted at 2026-08-07 19:38:49 on dist-test-slave-07c2"
server_key: "6764a1931ab65bf8278b98a75c505eb6"
server_key_iv: "767e715f255430c8533d815ff460ac5a"
server_key_version: "encryptionkey@0"
I20260807 19:38:49.354648 24660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:49.360378 24660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:49.361179 24660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:49.361454 24660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:49.362370 24660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:49.363850 24660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:49.363961 24660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:49.364095 24660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:49.364200 24660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:49.400858 24660 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:38691
I20260807 19:38:49.400935 24790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:38691 every 8 connection(s)
I20260807 19:38:49.402235 24660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:38:49.404441 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 24660
I20260807 19:38:49.404655 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:38:49.405325 14401 external_mini_cluster.cc:1428] Setting key 4d4e8bb9309c71d20da1b28d767a749c
I20260807 19:38:49.408603 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:46243
--builtin_ntp_servers=127.14.16.84:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 19:38:49.411551 24794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:49.405981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:36149 (local address 127.14.16.126:46243)
0807 19:38:49.407662 (+ 1681us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:49.407666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:49.407684 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:38:49.409021 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:49.409030 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:49.409110 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:49.409236 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:49.410631 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:49.410639 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:49.410646 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:49.410670 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:49.410728 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:49.410746 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:49.410903 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:49.411073 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 19:38:49.411146 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1590,"thread_start_us":1475,"threads_started":1}
I20260807 19:38:49.412358 24793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:49.405472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:46243 (local address 127.14.16.65:36149)
0807 19:38:49.406587 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:49.406649 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:49.408664 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:49.409440 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:49.409449 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:49.409540 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:49.410317 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:38:49.410320 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:49.410358 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:38:49.410447 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:49.410878 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:49.410886 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:49.410892 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:38:49.411108 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 19:38:49.411376 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":359,"threads_started":1}
I20260807 19:38:49.413921 24791 heartbeater.cc:347] Connected to a master server at 127.14.16.126:46243
I20260807 19:38:49.414222 24791 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:49.414835 24791 heartbeater.cc:511] Master 127.14.16.126:46243 requested a full tablet report, sending...
I20260807 19:38:49.417001 24598 ts_manager.cc:194] Registered new tserver with Master: b7295d2ce42840fbbd4116475cf8b462 (127.14.16.65:38691)
W20260807 19:38:49.754212 24795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:49.754485 24795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:49.754575 24795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:49.762145 24795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:49.762259 24795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:49.762444 24795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:38:49.771727 24795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:46243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:49.773098 24795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:49.774997 24795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:49.788585 24800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:49.788508 24803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:49.788656 24801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:49.789896 24795 server_base.cc:1034] running on GCE node
I20260807 19:38:49.790546 24795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:49.791625 24795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:49.792881 24795 hybrid_clock.cc:648] HybridClock initialized: now 1786131529792817 us; error 59 us; skew 500 ppm
I20260807 19:38:49.793210 24795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:38:49.799741 24795 webserver.cc:460] Webserver started at http://127.14.16.66:33367/ using document root <none> and password file <none>
I20260807 19:38:49.800597 24795 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:49.800729 24795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:49.801079 24795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:49.804059 24795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "91be17c53e8f4e138852e0b611e9e0c5"
format_stamp: "Formatted at 2026-08-07 19:38:49 on dist-test-slave-07c2"
server_key: "308ad887b5b4d22ea85565b5aacef944"
server_key_iv: "da3212e309cb06b5877c1d1548d452ca"
server_key_version: "encryptionkey@0"
I20260807 19:38:49.804838 24795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "91be17c53e8f4e138852e0b611e9e0c5"
format_stamp: "Formatted at 2026-08-07 19:38:49 on dist-test-slave-07c2"
server_key: "308ad887b5b4d22ea85565b5aacef944"
server_key_iv: "da3212e309cb06b5877c1d1548d452ca"
server_key_version: "encryptionkey@0"
I20260807 19:38:49.810134 24795 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:38:49.813434 24809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:49.814750 24795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:38:49.814915 24795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "91be17c53e8f4e138852e0b611e9e0c5"
format_stamp: "Formatted at 2026-08-07 19:38:49 on dist-test-slave-07c2"
server_key: "308ad887b5b4d22ea85565b5aacef944"
server_key_iv: "da3212e309cb06b5877c1d1548d452ca"
server_key_version: "encryptionkey@0"
I20260807 19:38:49.815141 24795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:49.821004 24795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:49.821810 24795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:49.822088 24795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:49.822973 24795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:49.824429 24795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:49.824540 24795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:49.824667 24795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:49.824764 24795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:49.858845 24795 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:43391
I20260807 19:38:49.858840 24924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:43391 every 8 connection(s)
I20260807 19:38:49.860255 24795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:38:49.868638 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 24795
I20260807 19:38:49.868778 24794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:49.862962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:48983 (local address 127.14.16.126:46243)
0807 19:38:49.863176 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:49.863183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:49.864797 (+ 1614us) server_negotiation.cc:407] Connection header received
0807 19:38:49.866471 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:49.866477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:49.866545 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:49.866668 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:49.868023 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:49.868029 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:49.868033 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:49.868052 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:49.868093 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:49.868106 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:49.868251 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:49.868430 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 19:38:49.868499 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:38:49.868894 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:38:49.869344 14401 external_mini_cluster.cc:1428] Setting key 1aa0f2ad9f9ef804827f4f9f80e4d36e
I20260807 19:38:49.869733 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:49.863597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:46243 (local address 127.14.16.66:48983)
0807 19:38:49.864640 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:49.864683 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:49.866205 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:49.866902 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:49.866910 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:49.866994 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:49.867721 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:38:49.867730 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:49.867766 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:38:49.867848 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:49.868204 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:49.868217 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:49.868226 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 19:38:49.868511 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 19:38:49.868772 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":335,"threads_started":1}
I20260807 19:38:49.871011 24925 heartbeater.cc:347] Connected to a master server at 127.14.16.126:46243
I20260807 19:38:49.871381 24925 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:49.871959 24925 heartbeater.cc:511] Master 127.14.16.126:46243 requested a full tablet report, sending...
I20260807 19:38:49.872272 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:46243
--builtin_ntp_servers=127.14.16.84:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 19:38:49.873358 24598 ts_manager.cc:194] Registered new tserver with Master: 91be17c53e8f4e138852e0b611e9e0c5 (127.14.16.66:43391)
W20260807 19:38:50.225646 24928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:50.225920 24928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:50.226011 24928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:50.233363 24928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:50.233461 24928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:50.233568 24928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:38:50.242354 24928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:46243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:50.243623 24928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:50.245325 24928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:50.258149 24933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:50.258114 24934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:50.258384 24936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:50.259353 24928 server_base.cc:1034] running on GCE node
I20260807 19:38:50.259984 24928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:50.260969 24928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:50.262197 24928 hybrid_clock.cc:648] HybridClock initialized: now 1786131530262158 us; error 56 us; skew 500 ppm
I20260807 19:38:50.262532 24928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:38:50.264905 24928 webserver.cc:460] Webserver started at http://127.14.16.67:36845/ using document root <none> and password file <none>
I20260807 19:38:50.265590 24928 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:50.265725 24928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:50.266058 24928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:50.269002 24928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "26a8ef168b094ef8a48ef0a867e6a721"
format_stamp: "Formatted at 2026-08-07 19:38:50 on dist-test-slave-07c2"
server_key: "378294ab60bb6b679ae4b3a2bc11d77a"
server_key_iv: "fde7839e4656025b762c3325859eea8f"
server_key_version: "encryptionkey@0"
I20260807 19:38:50.269816 24928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "26a8ef168b094ef8a48ef0a867e6a721"
format_stamp: "Formatted at 2026-08-07 19:38:50 on dist-test-slave-07c2"
server_key: "378294ab60bb6b679ae4b3a2bc11d77a"
server_key_iv: "fde7839e4656025b762c3325859eea8f"
server_key_version: "encryptionkey@0"
I20260807 19:38:50.274912 24928 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:38:50.278251 24942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:50.279587 24928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:38:50.279767 24928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "26a8ef168b094ef8a48ef0a867e6a721"
format_stamp: "Formatted at 2026-08-07 19:38:50 on dist-test-slave-07c2"
server_key: "378294ab60bb6b679ae4b3a2bc11d77a"
server_key_iv: "fde7839e4656025b762c3325859eea8f"
server_key_version: "encryptionkey@0"
I20260807 19:38:50.279992 24928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:50.285681 24928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:50.286480 24928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:50.286764 24928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:50.287794 24928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:50.289264 24928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:50.289388 24928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:50.289518 24928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:50.289605 24928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:50.324205 24928 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:42371
I20260807 19:38:50.324246 25057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:42371 every 8 connection(s)
I20260807 19:38:50.325531 24928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:38:50.330319 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 24928
I20260807 19:38:50.330521 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:38:50.331132 14401 external_mini_cluster.cc:1428] Setting key 1da8be814a91414db0ce9988963bfd50
I20260807 19:38:50.334451 24794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:50.328131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:53839 (local address 127.14.16.126:46243)
0807 19:38:50.328345 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:50.328352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:50.329789 (+ 1437us) server_negotiation.cc:407] Connection header received
0807 19:38:50.332142 (+ 2353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:50.332148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:50.332218 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:50.332329 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:50.333713 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:50.333717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:50.333720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:50.333750 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:50.333782 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:50.333793 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:50.333891 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:50.334129 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 19:38:50.334194 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:38:50.335480 25060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:50.328732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:46243 (local address 127.14.16.67:53839)
0807 19:38:50.329634 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:50.329679 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:50.331897 (+ 2218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:50.332572 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:50.332580 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:50.332665 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:50.333378 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:38:50.333384 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:50.333437 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:38:50.333545 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:50.333900 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:50.333910 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:50.333917 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 19:38:50.334231 (+ 314us) client_negotiation.cc:240] Negotiation successful
0807 19:38:50.334489 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":271,"threads_started":1}
I20260807 19:38:50.336863 25058 heartbeater.cc:347] Connected to a master server at 127.14.16.126:46243
I20260807 19:38:50.337174 25058 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:50.337692 25058 heartbeater.cc:511] Master 127.14.16.126:46243 requested a full tablet report, sending...
I20260807 19:38:50.339123 24598 ts_manager.cc:194] Registered new tserver with Master: 26a8ef168b094ef8a48ef0a867e6a721 (127.14.16.67:42371)
I20260807 19:38:50.346628 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 19:38:50.352931 24794 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:38:50.351424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58878 (local address 127.14.16.126:46243)
0807 19:38:50.351609 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:50.351613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:50.352150 (+ 537us) server_negotiation.cc:407] Connection header received
0807 19:38:50.352293 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:50.352297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:50.352349 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:50.352432 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:50.352735 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58878: BlockingRecv error: recv got EOF from 127.0.0.1:58878 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:38:50.353663 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 24660
I20260807 19:38:50.357537 25067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:50.354646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51072 (local address 127.14.16.65:38691)
0807 19:38:50.355170 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:50.355177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:50.355202 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:38:50.355258 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:50.355263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:50.355619 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:50.355768 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:50.356774 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:50.356782 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:50.356793 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:50.356809 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:50.356865 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:50.356884 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:50.356998 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:50.357079 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 19:38:50.357205 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":247,"threads_started":1}
I20260807 19:38:50.358781 24785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:50.442418 24791 heartbeater.cc:503] Master 127.14.16.126:46243 was elected leader, sending a full tablet report...
I20260807 19:38:50.443575 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 24660
W20260807 19:38:50.462797 24583 connection.cc:535] server connection from 127.14.16.65:36149 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 19:38:50.462913 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 24795
I20260807 19:38:50.466547 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:50.463926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44264 (local address 127.14.16.66:43391)
0807 19:38:50.464415 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:50.464421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:50.464448 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:38:50.464512 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:50.464518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:50.464754 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:50.464883 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:50.465947 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:50.465954 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:50.465965 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:50.465985 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:50.466027 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:50.466041 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:50.466144 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:50.466229 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 19:38:50.466307 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":229,"threads_started":1}
I20260807 19:38:50.467839 24920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:50.551532 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 24795
I20260807 19:38:50.570585 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 24928
I20260807 19:38:50.574750 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:50.571734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.14.16.67:42371)
0807 19:38:50.572445 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:50.572452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:50.572483 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:38:50.572546 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:50.572552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:50.572771 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:50.572895 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:50.573914 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:50.573920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:50.573931 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:38:50.573947 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:50.573986 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:38:50.573999 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:50.574133 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:50.574302 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 19:38:50.574423 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":372,"threads_started":1}
I20260807 19:38:50.576077 25053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:50.658600 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 24928
I20260807 19:38:50.677273 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 24566
I20260807 19:38:50.678503 24629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:50.781461 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 24566
2026-08-07T19:38:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2442 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:38:50 dist-test-slave-07c2 krb5kdc[25077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:38:50 dist-test-slave-07c2 krb5kdc[25077](info): set up 2 sockets
Aug 07 19:38:50 dist-test-slave-07c2 krb5kdc[25077](info): commencing operation
krb5kdc: starting...
W20260807 19:38:51.923636 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:38:52 dist-test-slave-07c2 krb5kdc[25077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:38:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:38:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:38:52.239450 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:40089
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:44083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:40089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:52.587162 25090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:52.587563 25090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:52.587728 25090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:52.595139 25090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:38:52.595330 25090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:52.595455 25090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:52.595556 25090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:38:52.595643 25090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:38:52.605785 25090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:44083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:40089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:40089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:52.607256 25090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:52.608652 25090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:52.620776 25095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:52.620776 25096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:52.620857 25098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:52.621595 25090 server_base.cc:1034] running on GCE node
I20260807 19:38:52.622437 25090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:52.623991 25090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:52.625234 25090 hybrid_clock.cc:648] HybridClock initialized: now 1786131532625159 us; error 47 us; skew 500 ppm
I20260807 19:38:52.625635 25090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:52 dist-test-slave-07c2 krb5kdc[25077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131532, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:52.636427 25090 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:38:52.638064 25090 webserver.cc:460] Webserver started at http://127.14.16.126:44513/ using document root <none> and password file <none>
I20260807 19:38:52.638758 25090 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:52.638860 25090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:52.639133 25090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:52.643803 25090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "de7c4b44136348ea891c39b7a96adb35"
format_stamp: "Formatted at 2026-08-07 19:38:52 on dist-test-slave-07c2"
server_key: "0859c1dfdd6cb88e8579d8d22edb522f"
server_key_iv: "5387e85b153f711f6882f5c0fb70ac18"
server_key_version: "encryptionkey@0"
I20260807 19:38:52.644937 25090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "de7c4b44136348ea891c39b7a96adb35"
format_stamp: "Formatted at 2026-08-07 19:38:52 on dist-test-slave-07c2"
server_key: "0859c1dfdd6cb88e8579d8d22edb522f"
server_key_iv: "5387e85b153f711f6882f5c0fb70ac18"
server_key_version: "encryptionkey@0"
I20260807 19:38:52.651961 25090 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 19:38:52.655668 25105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:52.657100 25090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:38:52.657315 25090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "de7c4b44136348ea891c39b7a96adb35"
format_stamp: "Formatted at 2026-08-07 19:38:52 on dist-test-slave-07c2"
server_key: "0859c1dfdd6cb88e8579d8d22edb522f"
server_key_iv: "5387e85b153f711f6882f5c0fb70ac18"
server_key_version: "encryptionkey@0"
I20260807 19:38:52.657526 25090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:52.675014 25090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:52.679375 25090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:52.679693 25090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:52.699602 25090 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:40089
I20260807 19:38:52.699633 25156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:40089 every 8 connection(s)
I20260807 19:38:52.701051 25090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:38:52.704846 25157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:52.709509 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 25090
I20260807 19:38:52.709780 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:38:52.710636 14401 external_mini_cluster.cc:1428] Setting key 2273ebf5f74692a4af53f2f804f17805
I20260807 19:38:52.717451 25157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35: Bootstrap starting.
I20260807 19:38:52.721240 25157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:52.722479 25157 log.cc:824] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:52.726223 25157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35: No bootstrap required, opened a new log
Aug 07 19:38:52 dist-test-slave-07c2 krb5kdc[25077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:38:52.734198 25157 raft_consensus.cc:348] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7c4b44136348ea891c39b7a96adb35" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40089 } }
I20260807 19:38:52.734689 25157 raft_consensus.cc:374] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:52.734828 25157 raft_consensus.cc:729] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de7c4b44136348ea891c39b7a96adb35, State: Initialized, Role: FOLLOWER
I20260807 19:38:52.735509 25157 consensus_queue.cc:260] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7c4b44136348ea891c39b7a96adb35" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40089 } }
I20260807 19:38:52.735755 25157 raft_consensus.cc:388] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:38:52.735918 25157 raft_consensus.cc:482] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:38:52.735958 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:52.712898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.14.16.126:40089)
0807 19:38:52.713739 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:52.713751 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:52.713793 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:38:52.715815 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:52.715843 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:52.716261 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:52.716606 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:52.718599 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:52.720107 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:52.721999 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:52.722746 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:52.722881 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:52.727966 (+ 5085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:52.727974 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:52.727991 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:52.728032 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:52.731365 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:52.732103 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:52.732118 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:52.732124 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:52.732308 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:52.732725 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:52.732730 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:52.732735 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:52.733344 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:52.733678 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:52.734133 (+ 455us) server_negotiation.cc:299] Negotiation successful
0807 19:38:52.734602 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":390,"threads_started":1}
I20260807 19:38:52.736104 25157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:52.738517 25157 raft_consensus.cc:504] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7c4b44136348ea891c39b7a96adb35" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40089 } }
I20260807 19:38:52.739108 25157 leader_election.cc:302] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de7c4b44136348ea891c39b7a96adb35; no voters:
I20260807 19:38:52.739753 25157 leader_election.cc:288] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:38:52.739905 25162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:52.740660 25162 raft_consensus.cc:686] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [term 1 LEADER]: Becoming Leader. State: Replica: de7c4b44136348ea891c39b7a96adb35, State: Running, Role: LEADER
I20260807 19:38:52.741199 25162 consensus_queue.cc:237] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7c4b44136348ea891c39b7a96adb35" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40089 } }
I20260807 19:38:52.741927 25157 sys_catalog.cc:564] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:38:52.744410 25164 sys_catalog.cc:455] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de7c4b44136348ea891c39b7a96adb35. Latest consensus state: current_term: 1 leader_uuid: "de7c4b44136348ea891c39b7a96adb35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7c4b44136348ea891c39b7a96adb35" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40089 } } }
I20260807 19:38:52.744869 25164 sys_catalog.cc:458] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:52.745363 25163 sys_catalog.cc:455] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de7c4b44136348ea891c39b7a96adb35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7c4b44136348ea891c39b7a96adb35" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40089 } } }
I20260807 19:38:52.745733 25163 sys_catalog.cc:458] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35 [sys.catalog]: This master's current role is: LEADER
I20260807 19:38:52.752152 25169 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:38:52.758102 25169 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:38:52.767683 25169 catalog_manager.cc:1347] Generated new cluster ID: 29bff1801b12462ba2ae9b5a4e983a85
I20260807 19:38:52.767830 25169 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:38:52.782042 25169 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:38:52.783664 25169 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:38:52.793910 25169 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35: Generated new TSK 0
I20260807 19:38:52.794987 25169 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:38:52.880985 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40089
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:44083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:53.250993 25184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:53.251371 25184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:53.251490 25184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:53.259147 25184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:53.259265 25184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:53.259452 25184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:38:53.269636 25184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:44083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:53.270942 25184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:53.272321 25184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:53.285740 25189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:53.285823 25190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:53.285847 25192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:53.291824 25184 server_base.cc:1034] running on GCE node
I20260807 19:38:53.292503 25184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:53.293646 25184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:53.294857 25184 hybrid_clock.cc:648] HybridClock initialized: now 1786131533294801 us; error 51 us; skew 500 ppm
I20260807 19:38:53.295204 25184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:53 dist-test-slave-07c2 krb5kdc[25077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131533, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:53.298766 25184 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:38:53.300453 25184 webserver.cc:460] Webserver started at http://127.14.16.65:35541/ using document root <none> and password file <none>
I20260807 19:38:53.301218 25184 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:53.301383 25184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:53.301703 25184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:53.304924 25184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "e17c59a60bfc4b64b7dcc986964a8373"
format_stamp: "Formatted at 2026-08-07 19:38:53 on dist-test-slave-07c2"
server_key: "b116182fef4d22af7700b85cd9f7e9ad"
server_key_iv: "5371cd0ae51043d0cd366766c57e18b2"
server_key_version: "encryptionkey@0"
I20260807 19:38:53.305704 25184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "e17c59a60bfc4b64b7dcc986964a8373"
format_stamp: "Formatted at 2026-08-07 19:38:53 on dist-test-slave-07c2"
server_key: "b116182fef4d22af7700b85cd9f7e9ad"
server_key_iv: "5371cd0ae51043d0cd366766c57e18b2"
server_key_version: "encryptionkey@0"
I20260807 19:38:53.311050 25184 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:38:53.314631 25199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:53.316013 25184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:38:53.316211 25184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "e17c59a60bfc4b64b7dcc986964a8373"
format_stamp: "Formatted at 2026-08-07 19:38:53 on dist-test-slave-07c2"
server_key: "b116182fef4d22af7700b85cd9f7e9ad"
server_key_iv: "5371cd0ae51043d0cd366766c57e18b2"
server_key_version: "encryptionkey@0"
I20260807 19:38:53.316428 25184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:53.341861 25184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:53.346447 25184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:53.346784 25184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:53.347852 25184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:53.349301 25184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:53.349409 25184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:53.349565 25184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:53.349637 25184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:53.384667 25184 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:39433
I20260807 19:38:53.384763 25312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:39433 every 8 connection(s)
I20260807 19:38:53.386035 25184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:38:53.395570 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 25184
I20260807 19:38:53.395816 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:38:53.396468 14401 external_mini_cluster.cc:1428] Setting key 9b3c3205c56708855d2a9276f3ddc387
Aug 07 19:38:53 dist-test-slave-07c2 krb5kdc[25077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131533, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:38:53.407254 25315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:53.388676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:43677 (local address 127.14.16.126:40089)
0807 19:38:53.389128 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:53.389132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:53.390092 (+ 960us) server_negotiation.cc:407] Connection header received
0807 19:38:53.392072 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:53.392078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:53.392183 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:53.392316 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:53.393779 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:53.394510 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:53.396279 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:53.396741 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:53.396880 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:53.400653 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:53.400659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:53.400662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:53.400695 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:53.403495 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:53.404316 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:53.404321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:53.404324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:53.404451 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:53.404935 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:53.404945 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:53.404947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:53.405229 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:53.405392 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:53.406468 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0807 19:38:53.406746 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":218,"threads_started":1}
I20260807 19:38:53.408295 25316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:53.389219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40089 (local address 127.14.16.65:43677)
0807 19:38:53.389941 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:53.389978 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:53.391806 (+ 1828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:53.392552 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:53.392560 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:53.393073 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:53.393602 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:53.393619 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:53.394685 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:53.394689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:53.396119 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:53.396132 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:53.396987 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:53.396994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:53.397269 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:53.398064 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:53.398102 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:53.400417 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:53.403653 (+ 3236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:53.403663 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:53.403683 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:53.404164 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:53.404571 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:53.404577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:53.404580 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:53.404804 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:53.405368 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:53.405376 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:53.405922 (+ 546us) client_negotiation.cc:769] Sending connection context
0807 19:38:53.406306 (+ 384us) client_negotiation.cc:240] Negotiation successful
0807 19:38:53.406916 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":276,"threads_started":1}
I20260807 19:38:53.410001 25313 heartbeater.cc:347] Connected to a master server at 127.14.16.126:40089
I20260807 19:38:53.410337 25313 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:53.411082 25313 heartbeater.cc:511] Master 127.14.16.126:40089 requested a full tablet report, sending...
I20260807 19:38:53.413414 25122 ts_manager.cc:194] Registered new tserver with Master: e17c59a60bfc4b64b7dcc986964a8373 (127.14.16.65:39433)
I20260807 19:38:53.414952 25122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:43677
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:38:53.460667 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40089
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:44083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:53.809561 25319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:53.809855 25319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:53.810009 25319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:53.817409 25319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:53.817510 25319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:53.817713 25319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:38:53.827342 25319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:44083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:53.828646 25319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:53.829911 25319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:53.842813 25327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:53.842861 25325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:53.842774 25324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:53.848776 25319 server_base.cc:1034] running on GCE node
I20260807 19:38:53.849409 25319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:53.850400 25319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:53.851626 25319 hybrid_clock.cc:648] HybridClock initialized: now 1786131533851587 us; error 52 us; skew 500 ppm
I20260807 19:38:53.851940 25319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:53 dist-test-slave-07c2 krb5kdc[25077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131533, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:53.855139 25319 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:38:53.856657 25319 webserver.cc:460] Webserver started at http://127.14.16.66:34853/ using document root <none> and password file <none>
I20260807 19:38:53.857332 25319 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:53.857436 25319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:53.857697 25319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:53.860670 25319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "c920e47e638f4bad87efd47190dcc0ab"
format_stamp: "Formatted at 2026-08-07 19:38:53 on dist-test-slave-07c2"
server_key: "30498d81fd1ef6b3f212ccc073ea243e"
server_key_iv: "c66b9c26daa513139c57658774d797e4"
server_key_version: "encryptionkey@0"
I20260807 19:38:53.861459 25319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "c920e47e638f4bad87efd47190dcc0ab"
format_stamp: "Formatted at 2026-08-07 19:38:53 on dist-test-slave-07c2"
server_key: "30498d81fd1ef6b3f212ccc073ea243e"
server_key_iv: "c66b9c26daa513139c57658774d797e4"
server_key_version: "encryptionkey@0"
I20260807 19:38:53.866513 25319 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 19:38:53.869987 25334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:53.871425 25319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:38:53.871599 25319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "c920e47e638f4bad87efd47190dcc0ab"
format_stamp: "Formatted at 2026-08-07 19:38:53 on dist-test-slave-07c2"
server_key: "30498d81fd1ef6b3f212ccc073ea243e"
server_key_iv: "c66b9c26daa513139c57658774d797e4"
server_key_version: "encryptionkey@0"
I20260807 19:38:53.871811 25319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:53.908004 25319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:53.912715 25319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:53.913023 25319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:53.913990 25319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:53.915405 25319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:53.915505 25319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:53.915664 25319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:53.915745 25319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:53.949509 25319 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:38613
I20260807 19:38:53.949602 25450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:38613 every 8 connection(s)
I20260807 19:38:53.950888 25319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:38:53.951915 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 25319
I20260807 19:38:53.952126 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:38:53.952690 14401 external_mini_cluster.cc:1428] Setting key 1a63a7abd734dc99d838e6ea59c00e14
Aug 07 19:38:53 dist-test-slave-07c2 krb5kdc[25077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131533, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:38:53.973088 25453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:53.953866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:37079 (local address 127.14.16.126:40089)
0807 19:38:53.954382 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:53.954388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:53.955549 (+ 1161us) server_negotiation.cc:407] Connection header received
0807 19:38:53.957420 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:53.957426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:53.957524 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:53.957647 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:53.959210 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:53.960030 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:53.961631 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:53.962168 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:53.962309 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:53.965905 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:53.965910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:53.965913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:53.965948 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:53.969558 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:53.970333 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:53.970342 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:53.970348 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:53.970546 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:53.971000 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:53.971007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:53.971013 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:53.971441 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:53.971600 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:53.972319 (+ 719us) server_negotiation.cc:299] Negotiation successful
0807 19:38:53.972591 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":275,"threads_started":1}
I20260807 19:38:53.973784 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:53.954542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40089 (local address 127.14.16.66:37079)
0807 19:38:53.955393 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:53.955432 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:53.957126 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:53.957907 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:53.957916 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:53.958461 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:53.959025 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:53.959042 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:53.960187 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:53.960194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:53.961466 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:53.961477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:53.962417 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:53.962425 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:53.962656 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:53.963435 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:53.963464 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:53.965656 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:53.969725 (+ 4069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:53.969733 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:53.969747 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:53.970195 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:53.970686 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:53.970690 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:53.970692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:53.970882 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:53.971570 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:53.971576 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:53.971911 (+ 335us) client_negotiation.cc:769] Sending connection context
0807 19:38:53.972159 (+ 248us) client_negotiation.cc:240] Negotiation successful
0807 19:38:53.972588 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":343,"threads_started":1}
I20260807 19:38:53.975409 25451 heartbeater.cc:347] Connected to a master server at 127.14.16.126:40089
I20260807 19:38:53.975800 25451 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:53.976737 25451 heartbeater.cc:511] Master 127.14.16.126:40089 requested a full tablet report, sending...
I20260807 19:38:53.978485 25122 ts_manager.cc:194] Registered new tserver with Master: c920e47e638f4bad87efd47190dcc0ab (127.14.16.66:38613)
I20260807 19:38:53.979414 25122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:37079
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:38:54.024926 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40089
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:44083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:38:54.366782 25457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:38:54.367077 25457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:38:54.367244 25457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:38:54.374693 25457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:38:54.374802 25457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:38:54.374977 25457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:38:54.385006 25457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:44083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:38:54.386307 25457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:38:54.387683 25457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:38:54.401470 25462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:54.401437 25465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:38:54.401510 25463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:38:54.402701 25457 server_base.cc:1034] running on GCE node
I20260807 19:38:54.403448 25457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:38:54.404410 25457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:38:54.405650 25457 hybrid_clock.cc:648] HybridClock initialized: now 1786131534405567 us; error 54 us; skew 500 ppm
I20260807 19:38:54.405939 25457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:38:54 dist-test-slave-07c2 krb5kdc[25077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131534, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:38:54.409689 25457 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:38:54.412041 25457 webserver.cc:460] Webserver started at http://127.14.16.67:36467/ using document root <none> and password file <none>
I20260807 19:38:54.412827 25457 fs_manager.cc:359] Metadata directory not provided
I20260807 19:38:54.412963 25457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:38:54.413300 25457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:38:54.416486 25457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "132a6610833a4348aea0c23a989f5dec"
format_stamp: "Formatted at 2026-08-07 19:38:54 on dist-test-slave-07c2"
server_key: "fea74eaf7730b07c993cfcdee8200113"
server_key_iv: "3f648ecc2bca86ec22dadda6cf0b55df"
server_key_version: "encryptionkey@0"
I20260807 19:38:54.417300 25457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "132a6610833a4348aea0c23a989f5dec"
format_stamp: "Formatted at 2026-08-07 19:38:54 on dist-test-slave-07c2"
server_key: "fea74eaf7730b07c993cfcdee8200113"
server_key_iv: "3f648ecc2bca86ec22dadda6cf0b55df"
server_key_version: "encryptionkey@0"
I20260807 19:38:54.418893 25313 heartbeater.cc:503] Master 127.14.16.126:40089 was elected leader, sending a full tablet report...
I20260807 19:38:54.423578 25457 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 19:38:54.427542 25473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:54.429060 25457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:38:54.429277 25457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "132a6610833a4348aea0c23a989f5dec"
format_stamp: "Formatted at 2026-08-07 19:38:54 on dist-test-slave-07c2"
server_key: "fea74eaf7730b07c993cfcdee8200113"
server_key_iv: "3f648ecc2bca86ec22dadda6cf0b55df"
server_key_version: "encryptionkey@0"
I20260807 19:38:54.429497 25457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:38:54.448644 25457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:38:54.453438 25457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:38:54.453794 25457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:38:54.454854 25457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:38:54.456312 25457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:38:54.456391 25457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:54.456558 25457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:38:54.456616 25457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:38:54.492364 25457 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:43809
I20260807 19:38:54.492411 25592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:43809 every 8 connection(s)
I20260807 19:38:54.493777 25457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:38:54.495220 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 25457
I20260807 19:38:54.495473 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:38:54.496032 14401 external_mini_cluster.cc:1428] Setting key d48d64855d1a9a56b316d6f4c20a2b39
Aug 07 19:38:54 dist-test-slave-07c2 krb5kdc[25077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131534, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:38:54.515214 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.497119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:45241 (local address 127.14.16.126:40089)
0807 19:38:54.498230 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:54.498239 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:54.498807 (+ 568us) server_negotiation.cc:407] Connection header received
0807 19:38:54.500840 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:54.500844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:54.500913 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:54.501003 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:54.502512 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.503447 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.505020 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.505463 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.505570 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.509314 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:54.509319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:54.509322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:54.509360 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:54.512459 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:54.513194 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:54.513198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:54.513200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:54.513333 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:54.513720 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:54.513723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:54.513725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:54.513986 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:54.514103 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:54.514764 (+ 661us) server_negotiation.cc:299] Negotiation successful
0807 19:38:54.514936 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":958,"thread_start_us":463,"threads_started":1}
I20260807 19:38:54.516307 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.497749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40089 (local address 127.14.16.67:45241)
0807 19:38:54.498628 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:54.498679 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:54.500585 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:54.501211 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:54.501219 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:54.501743 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:38:54.502347 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:54.502364 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.503600 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.503604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:54.504856 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:54.504869 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.505710 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.505716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:54.505949 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.506628 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:38:54.506655 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:38:54.509069 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:38:54.512608 (+ 3539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:54.512616 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:54.512631 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:54.513090 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:54.513429 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:38:54.513432 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:38:54.513435 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:38:54.513621 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:38:54.514091 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:38:54.514097 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:38:54.514435 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 19:38:54.514676 (+ 241us) client_negotiation.cc:240] Negotiation successful
0807 19:38:54.515112 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":382,"threads_started":1}
I20260807 19:38:54.517777 25593 heartbeater.cc:347] Connected to a master server at 127.14.16.126:40089
I20260807 19:38:54.518051 25593 heartbeater.cc:464] Registering TS with master...
I20260807 19:38:54.518749 25593 heartbeater.cc:511] Master 127.14.16.126:40089 requested a full tablet report, sending...
I20260807 19:38:54.520254 25122 ts_manager.cc:194] Registered new tserver with Master: 132a6610833a4348aea0c23a989f5dec (127.14.16.67:43809)
I20260807 19:38:54.521134 25122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:45241
I20260807 19:38:54.524783 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:38:54.542639 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.529375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44084 (local address 127.14.16.126:40089)
0807 19:38:54.529585 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:54.529591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:54.530073 (+ 482us) server_negotiation.cc:407] Connection header received
0807 19:38:54.530196 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:54.530201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:54.530279 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:54.530373 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:54.532114 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.532989 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.534559 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.534981 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.535084 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.537344 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:54.537349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:54.537352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:54.537386 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:54.540171 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:54.540815 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:54.540819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:54.540821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:54.540945 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:54.541282 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:54.541285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:54.541287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:54.541606 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:54.541789 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:54.542151 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 19:38:54.542392 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:38:54.547114 25122 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:38:54.554440 25122 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:38:54.583994 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.568458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50218 (local address 127.14.16.65:39433)
0807 19:38:54.569196 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:54.569208 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:54.570679 (+ 1471us) server_negotiation.cc:407] Connection header received
0807 19:38:54.571234 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:54.571247 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:54.571632 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:54.571894 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:54.573931 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.576390 (+ 2459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.580754 (+ 4364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.582523 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.582699 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.582988 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:54.583375 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 19:38:54.583579 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":962,"threads_started":1}
I20260807 19:38:54.584932 25608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.568366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39433 (local address 127.0.0.1:50218)
0807 19:38:54.570465 (+ 2099us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:54.570531 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:54.570894 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:54.572929 (+ 2035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:54.572941 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:54.572987 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:54.573602 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:54.573622 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.576708 (+ 3086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.576717 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:54.580468 (+ 3751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:54.580490 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.582755 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.582770 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:54.583009 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.583121 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 19:38:54.583306 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 19:38:54.583664 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1711,"mutex_wait_us":208,"thread_start_us":10821,"threads_started":1}
I20260807 19:38:54.588824 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.569094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:43809 (local address 127.0.0.1:46848)
0807 19:38:54.578812 (+ 9718us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:54.578838 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:54.579026 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:54.580064 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:54.580069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:54.580092 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:54.580708 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:54.580724 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.582935 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.582943 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:54.585778 (+ 2835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:54.585793 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.587880 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.587887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:54.588073 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.588107 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:38:54.588215 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 19:38:54.588363 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9594,"mutex_wait_us":80,"thread_start_us":763,"threads_started":1}
I20260807 19:38:54.589371 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.569471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.14.16.67:43809)
0807 19:38:54.578360 (+ 8889us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:54.578368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:54.578970 (+ 602us) server_negotiation.cc:407] Connection header received
0807 19:38:54.579231 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:54.579242 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:54.579593 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:54.579855 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:54.580928 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.582762 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.585967 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.587654 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.587816 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.588100 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:54.588285 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 19:38:54.588976 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8689,"thread_start_us":8508,"threads_started":1}
I20260807 19:38:54.589452 25611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.572663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:38613 (local address 127.0.0.1:58116)
0807 19:38:54.579759 (+ 7096us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:54.579792 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:54.579908 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:54.581407 (+ 1499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:54.581411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:54.581441 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:54.581799 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:54.581808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.583455 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.583460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:54.586045 (+ 2585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:54.586060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.588399 (+ 2339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.588407 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:54.588621 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.588658 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:38:54.588773 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 19:38:54.588927 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6945,"mutex_wait_us":3,"thread_start_us":6764,"threads_started":1}
I20260807 19:38:54.589129 25248 tablet_service.cc:1467] Processing CreateTablet for tablet bd300abe87414e728b50207b039ef62e (DEFAULT_TABLE table=test-table [id=7dbc633c6fdc444588b361d4d7d03558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:54.590224 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.571228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.14.16.66:38613)
0807 19:38:54.580624 (+ 9396us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:54.580635 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:54.580665 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:38:54.580765 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:54.580773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:54.581021 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:54.581237 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:54.581972 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.583208 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.586239 (+ 3031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.587847 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.587989 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.588226 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:54.589677 (+ 1451us) server_negotiation.cc:299] Negotiation successful
0807 19:38:54.589893 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9209,"thread_start_us":9390,"threads_started":1}
I20260807 19:38:54.590911 25248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd300abe87414e728b50207b039ef62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:54.594134 25384 tablet_service.cc:1467] Processing CreateTablet for tablet bd300abe87414e728b50207b039ef62e (DEFAULT_TABLE table=test-table [id=7dbc633c6fdc444588b361d4d7d03558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:54.594133 25524 tablet_service.cc:1467] Processing CreateTablet for tablet bd300abe87414e728b50207b039ef62e (DEFAULT_TABLE table=test-table [id=7dbc633c6fdc444588b361d4d7d03558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:38:54.595810 25384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd300abe87414e728b50207b039ef62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:54.596393 25524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd300abe87414e728b50207b039ef62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:38:54.603753 25612 tablet_bootstrap.cc:492] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373: Bootstrap starting.
I20260807 19:38:54.606890 25612 tablet_bootstrap.cc:654] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:54.608052 25612 log.cc:824] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:54.611827 25614 tablet_bootstrap.cc:492] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab: Bootstrap starting.
I20260807 19:38:54.613855 25612 tablet_bootstrap.cc:492] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373: No bootstrap required, opened a new log
I20260807 19:38:54.614140 25612 ts_tablet_manager.cc:1397] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 19:38:54.615713 25614 tablet_bootstrap.cc:654] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:54.617023 25614 log.cc:824] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:54.618342 25615 tablet_bootstrap.cc:492] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec: Bootstrap starting.
I20260807 19:38:54.622217 25615 tablet_bootstrap.cc:654] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec: Neither blocks nor log segments found. Creating new log.
I20260807 19:38:54.623595 25615 log.cc:824] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec: Log is configured to *not* fsync() on all Append() calls
I20260807 19:38:54.624022 25614 tablet_bootstrap.cc:492] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab: No bootstrap required, opened a new log
I20260807 19:38:54.624404 25614 ts_tablet_manager.cc:1397] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260807 19:38:54.625311 25612 raft_consensus.cc:348] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17c59a60bfc4b64b7dcc986964a8373" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39433 } } peers { permanent_uuid: "132a6610833a4348aea0c23a989f5dec" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 43809 } } peers { permanent_uuid: "c920e47e638f4bad87efd47190dcc0ab" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 38613 } }
I20260807 19:38:54.626278 25612 raft_consensus.cc:374] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:54.626415 25612 raft_consensus.cc:729] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17c59a60bfc4b64b7dcc986964a8373, State: Initialized, Role: FOLLOWER
I20260807 19:38:54.627177 25615 tablet_bootstrap.cc:492] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec: No bootstrap required, opened a new log
I20260807 19:38:54.627113 25612 consensus_queue.cc:260] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17c59a60bfc4b64b7dcc986964a8373" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39433 } } peers { permanent_uuid: "132a6610833a4348aea0c23a989f5dec" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 43809 } } peers { permanent_uuid: "c920e47e638f4bad87efd47190dcc0ab" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 38613 } }
I20260807 19:38:54.627547 25615 ts_tablet_manager.cc:1397] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260807 19:38:54.629251 25612 ts_tablet_manager.cc:1428] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373: Time spent starting tablet: real 0.015s user 0.007s sys 0.008s
I20260807 19:38:54.634442 25614 raft_consensus.cc:348] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17c59a60bfc4b64b7dcc986964a8373" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39433 } } peers { permanent_uuid: "132a6610833a4348aea0c23a989f5dec" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 43809 } } peers { permanent_uuid: "c920e47e638f4bad87efd47190dcc0ab" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 38613 } }
I20260807 19:38:54.634860 25614 raft_consensus.cc:374] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:54.634991 25614 raft_consensus.cc:729] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c920e47e638f4bad87efd47190dcc0ab, State: Initialized, Role: FOLLOWER
I20260807 19:38:54.635785 25614 consensus_queue.cc:260] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17c59a60bfc4b64b7dcc986964a8373" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39433 } } peers { permanent_uuid: "132a6610833a4348aea0c23a989f5dec" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 43809 } } peers { permanent_uuid: "c920e47e638f4bad87efd47190dcc0ab" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 38613 } }
I20260807 19:38:54.636929 25451 heartbeater.cc:503] Master 127.14.16.126:40089 was elected leader, sending a full tablet report...
I20260807 19:38:54.637676 25614 ts_tablet_manager.cc:1428] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 19:38:54.637796 25615 raft_consensus.cc:348] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17c59a60bfc4b64b7dcc986964a8373" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39433 } } peers { permanent_uuid: "132a6610833a4348aea0c23a989f5dec" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 43809 } } peers { permanent_uuid: "c920e47e638f4bad87efd47190dcc0ab" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 38613 } }
I20260807 19:38:54.638250 25615 raft_consensus.cc:374] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:38:54.638377 25615 raft_consensus.cc:729] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 132a6610833a4348aea0c23a989f5dec, State: Initialized, Role: FOLLOWER
W20260807 19:38:54.638432 25314 tablet.cc:2369] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:38:54.639039 25615 consensus_queue.cc:260] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17c59a60bfc4b64b7dcc986964a8373" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39433 } } peers { permanent_uuid: "132a6610833a4348aea0c23a989f5dec" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 43809 } } peers { permanent_uuid: "c920e47e638f4bad87efd47190dcc0ab" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 38613 } }
I20260807 19:38:54.640282 25593 heartbeater.cc:503] Master 127.14.16.126:40089 was elected leader, sending a full tablet report...
I20260807 19:38:54.640861 25615 ts_tablet_manager.cc:1428] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 19:38:54.646911 25619 raft_consensus.cc:482] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:38:54.647224 25619 raft_consensus.cc:504] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17c59a60bfc4b64b7dcc986964a8373" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39433 } } peers { permanent_uuid: "132a6610833a4348aea0c23a989f5dec" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 43809 } } peers { permanent_uuid: "c920e47e638f4bad87efd47190dcc0ab" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 38613 } }
I20260807 19:38:54.648979 25619 leader_election.cc:288] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e17c59a60bfc4b64b7dcc986964a8373 (127.14.16.65:39433), 132a6610833a4348aea0c23a989f5dec (127.14.16.67:43809)
I20260807 19:38:54.656260 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.649300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:49495 (local address 127.14.16.65:39433)
0807 19:38:54.649527 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:54.649535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:54.650485 (+ 950us) server_negotiation.cc:407] Connection header received
0807 19:38:54.650669 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:54.650673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:54.650748 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:54.650857 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:54.651470 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.652219 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.654242 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.655131 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.655259 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.655401 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:54.655853 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 19:38:54.655985 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:38:54.656281 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.649500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:42173 (local address 127.14.16.67:43809)
0807 19:38:54.649731 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:54.649740 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:54.650340 (+ 600us) server_negotiation.cc:407] Connection header received
0807 19:38:54.650485 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:54.650491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:54.650567 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:54.650671 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:38:54.651329 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.652143 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.654319 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.655239 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.655463 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.655545 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:54.655897 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 19:38:54.656027 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:38:54.656446 25621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.649323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39433 (local address 127.14.16.66:49495)
0807 19:38:54.650394 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:54.650410 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:54.650540 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:54.650887 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:54.650890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:54.650909 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:54.651321 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:54.651332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.652357 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.652361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:54.654081 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:54.654095 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.655414 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.655421 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:54.655676 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.655718 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 19:38:54.655794 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 19:38:54.655943 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"mutex_wait_us":67,"thread_start_us":521,"threads_started":1}
I20260807 19:38:54.656700 25622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.649291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:43809 (local address 127.14.16.66:42173)
0807 19:38:54.650227 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0807 19:38:54.650244 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:38:54.650367 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:38:54.650691 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:38:54.650704 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:38:54.650727 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:38:54.651161 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:54.651175 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.652307 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.652313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:54.654098 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:38:54.654110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.655509 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.655514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:38:54.655717 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.655740 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:38:54.655852 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 19:38:54.655985 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"mutex_wait_us":103,"thread_start_us":739,"threads_started":1}
I20260807 19:38:54.657501 25268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd300abe87414e728b50207b039ef62e" candidate_uuid: "c920e47e638f4bad87efd47190dcc0ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17c59a60bfc4b64b7dcc986964a8373" is_pre_election: true
I20260807 19:38:54.657742 25545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd300abe87414e728b50207b039ef62e" candidate_uuid: "c920e47e638f4bad87efd47190dcc0ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "132a6610833a4348aea0c23a989f5dec" is_pre_election: true
I20260807 19:38:54.658072 25268 raft_consensus.cc:2454] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c920e47e638f4bad87efd47190dcc0ab in term 0.
I20260807 19:38:54.658299 25545 raft_consensus.cc:2454] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c920e47e638f4bad87efd47190dcc0ab in term 0.
I20260807 19:38:54.658828 25338 leader_election.cc:302] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c920e47e638f4bad87efd47190dcc0ab, e17c59a60bfc4b64b7dcc986964a8373; no voters:
I20260807 19:38:54.659305 25619 raft_consensus.cc:2781] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:38:54.659461 25619 raft_consensus.cc:482] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:38:54.659574 25619 raft_consensus.cc:3037] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:54.662045 25619 raft_consensus.cc:504] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17c59a60bfc4b64b7dcc986964a8373" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39433 } } peers { permanent_uuid: "132a6610833a4348aea0c23a989f5dec" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 43809 } } peers { permanent_uuid: "c920e47e638f4bad87efd47190dcc0ab" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 38613 } }
I20260807 19:38:54.662731 25619 leader_election.cc:288] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [CANDIDATE]: Term 1 election: Requested vote from peers e17c59a60bfc4b64b7dcc986964a8373 (127.14.16.65:39433), 132a6610833a4348aea0c23a989f5dec (127.14.16.67:43809)
I20260807 19:38:54.663271 25268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd300abe87414e728b50207b039ef62e" candidate_uuid: "c920e47e638f4bad87efd47190dcc0ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17c59a60bfc4b64b7dcc986964a8373"
I20260807 19:38:54.663444 25545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd300abe87414e728b50207b039ef62e" candidate_uuid: "c920e47e638f4bad87efd47190dcc0ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "132a6610833a4348aea0c23a989f5dec"
I20260807 19:38:54.663537 25268 raft_consensus.cc:3037] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:54.663700 25545 raft_consensus.cc:3037] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:38:54.665871 25268 raft_consensus.cc:2454] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c920e47e638f4bad87efd47190dcc0ab in term 1.
I20260807 19:38:54.666153 25545 raft_consensus.cc:2454] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c920e47e638f4bad87efd47190dcc0ab in term 1.
I20260807 19:38:54.666384 25338 leader_election.cc:302] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c920e47e638f4bad87efd47190dcc0ab, e17c59a60bfc4b64b7dcc986964a8373; no voters:
I20260807 19:38:54.666733 25619 raft_consensus.cc:2781] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:38:54.667222 25619 raft_consensus.cc:686] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 1 LEADER]: Becoming Leader. State: Replica: c920e47e638f4bad87efd47190dcc0ab, State: Running, Role: LEADER
I20260807 19:38:54.667833 25619 consensus_queue.cc:237] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17c59a60bfc4b64b7dcc986964a8373" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39433 } } peers { permanent_uuid: "132a6610833a4348aea0c23a989f5dec" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 43809 } } peers { permanent_uuid: "c920e47e638f4bad87efd47190dcc0ab" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 38613 } }
I20260807 19:38:54.672708 25122 catalog_manager.cc:5495] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab reported cstate change: term changed from 0 to 1, leader changed from <none> to c920e47e638f4bad87efd47190dcc0ab (127.14.16.66). New cstate: current_term: 1 leader_uuid: "c920e47e638f4bad87efd47190dcc0ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e17c59a60bfc4b64b7dcc986964a8373" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "132a6610833a4348aea0c23a989f5dec" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 43809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c920e47e638f4bad87efd47190dcc0ab" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260807 19:38:54.691601 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.686421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58130 (local address 127.14.16.66:38613)
0807 19:38:54.686615 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:54.686620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:54.686643 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:38:54.686735 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:54.686738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:54.686816 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:54.686949 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:38:54.687281 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.688044 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.689351 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.689762 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.689857 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.690427 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:38:54.690983 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:38:54.691096 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:54.691239 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 19:38:54.691372 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:38:54.700198 25545 raft_consensus.cc:1261] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec [term 1 FOLLOWER]: Refusing update from remote peer c920e47e638f4bad87efd47190dcc0ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:54.700199 25268 raft_consensus.cc:1261] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373 [term 1 FOLLOWER]: Refusing update from remote peer c920e47e638f4bad87efd47190dcc0ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:38:54.701283 25624 consensus_queue.cc:1035] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17c59a60bfc4b64b7dcc986964a8373" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:38:54.701659 25619 consensus_queue.cc:1035] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "132a6610833a4348aea0c23a989f5dec" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 43809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:38:54.703120 25452 tablet.cc:2369] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:38:54.717509 25627 mvcc.cc:205] Tried to move back new op lower bound from 7315994766121070592 to 7315994766003941376. Current Snapshot: MvccSnapshot[applied={T|T < 7315994766121070592}]
I20260807 19:38:54.719045 25630 mvcc.cc:205] Tried to move back new op lower bound from 7315994766121070592 to 7315994766003941376. Current Snapshot: MvccSnapshot[applied={T|T < 7315994766121070592}]
I20260807 19:38:54.722615 25629 mvcc.cc:205] Tried to move back new op lower bound from 7315994766121070592 to 7315994766003941376. Current Snapshot: MvccSnapshot[applied={T|T < 7315994766121070592}]
I20260807 19:38:54.734345 25122 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:54.734627 25122 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:38:54.738440 25122 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de7c4b44136348ea891c39b7a96adb35: Sending DeleteTablet for 3 replicas of tablet bd300abe87414e728b50207b039ef62e
I20260807 19:38:54.739771 25384 tablet_service.cc:1514] Processing DeleteTablet for tablet bd300abe87414e728b50207b039ef62e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:54 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:58116
I20260807 19:38:54.739699 25248 tablet_service.cc:1514] Processing DeleteTablet for tablet bd300abe87414e728b50207b039ef62e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:54 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:50218
I20260807 19:38:54.740118 25524 tablet_service.cc:1514] Processing DeleteTablet for tablet bd300abe87414e728b50207b039ef62e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:38:54 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:46848
I20260807 19:38:54.741257 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 25184
I20260807 19:38:54.750981 25639 tablet_replica.cc:330] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec: stopping tablet replica
I20260807 19:38:54.751585 25639 raft_consensus.cc:2229] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:54.752233 25639 raft_consensus.cc:2258] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:54.753243 25638 tablet_replica.cc:330] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab: stopping tablet replica
I20260807 19:38:54.754068 25637 tablet_replica.cc:330] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373: stopping tablet replica
I20260807 19:38:54.754469 25637 raft_consensus.cc:2229] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:38:54.754999 25637 raft_consensus.cc:2258] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:54.755477 25638 raft_consensus.cc:2229] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:38:54.756716 25639 ts_tablet_manager.cc:1905] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:54.757045 25637 ts_tablet_manager.cc:1905] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:54.757560 25638 raft_consensus.cc:2258] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:38:54.763181 25638 ts_tablet_manager.cc:1905] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:38:54.764750 25637 ts_tablet_manager.cc:1918] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:54.764925 25637 log.cc:1196] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/bd300abe87414e728b50207b039ef62e
Aug 07 19:38:54 dist-test-slave-07c2 krb5kdc[25077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:38:54.765470 25637 ts_tablet_manager.cc:1939] T bd300abe87414e728b50207b039ef62e P e17c59a60bfc4b64b7dcc986964a8373: Deleting consensus metadata
I20260807 19:38:54.767834 25109 catalog_manager.cc:4841] TS e17c59a60bfc4b64b7dcc986964a8373 (127.14.16.65:39433): tablet bd300abe87414e728b50207b039ef62e (table test-table [id=7dbc633c6fdc444588b361d4d7d03558]) successfully deleted
I20260807 19:38:54.772122 25639 ts_tablet_manager.cc:1918] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:54.772326 25639 log.cc:1196] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/bd300abe87414e728b50207b039ef62e
I20260807 19:38:54.772921 25639 ts_tablet_manager.cc:1939] T bd300abe87414e728b50207b039ef62e P 132a6610833a4348aea0c23a989f5dec: Deleting consensus metadata
I20260807 19:38:54.773324 25638 ts_tablet_manager.cc:1918] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:38:54.773571 25638 log.cc:1196] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/bd300abe87414e728b50207b039ef62e
I20260807 19:38:54.774232 25638 ts_tablet_manager.cc:1939] T bd300abe87414e728b50207b039ef62e P c920e47e638f4bad87efd47190dcc0ab: Deleting consensus metadata
I20260807 19:38:54.774639 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.743117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50228 (local address 127.14.16.65:39433)
0807 19:38:54.743398 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:54.743407 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:54.754297 (+ 10890us) server_negotiation.cc:407] Connection header received
0807 19:38:54.754598 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:54.754604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:54.754701 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:54.754813 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:54.756693 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.757861 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.759817 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.760542 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.760655 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.766543 (+ 5888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:54.766559 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:54.766577 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:54.766625 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:54.770737 (+ 4112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:54.771782 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:54.771792 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:54.771796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:54.771974 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:54.772458 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:54.772463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:54.772466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:54.772941 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:54.773269 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:54.773948 (+ 679us) server_negotiation.cc:299] Negotiation successful
0807 19:38:54.774210 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 19:38:54.775013 25108 catalog_manager.cc:4841] TS 132a6610833a4348aea0c23a989f5dec (127.14.16.67:43809): tablet bd300abe87414e728b50207b039ef62e (table test-table [id=7dbc633c6fdc444588b361d4d7d03558]) successfully deleted
I20260807 19:38:54.776695 25308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:54.776644 25108 catalog_manager.cc:4841] TS c920e47e638f4bad87efd47190dcc0ab (127.14.16.66:38613): tablet bd300abe87414e728b50207b039ef62e (table test-table [id=7dbc633c6fdc444588b361d4d7d03558]) successfully deleted
I20260807 19:38:54.869692 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 25184
I20260807 19:38:54.897259 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 25319
Aug 07 19:38:54 dist-test-slave-07c2 krb5kdc[25077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:38:54.915498 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:54.898427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58140 (local address 127.14.16.66:38613)
0807 19:38:54.898607 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:54.898611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:54.898635 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:38:54.898712 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:54.898716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:54.898785 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:54.898866 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:54.900630 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.901519 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.903433 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:54.903891 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:54.904006 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:54.909094 (+ 5088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:54.909112 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:54.909128 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:54.909171 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:54.912481 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:54.913209 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:54.913216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:54.913219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:54.913373 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:54.913789 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:54.913793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:54.913795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:54.914156 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:54.914543 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:54.914822 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 19:38:54.915054 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:38:54.916234 25446 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:55.030925 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 25319
I20260807 19:38:55.052886 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 25457
Aug 07 19:38:55 dist-test-slave-07c2 krb5kdc[25077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:38:55.068596 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:38:55.053799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46856 (local address 127.14.16.67:43809)
0807 19:38:55.053988 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:38:55.053992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:38:55.054033 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:38:55.054286 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:38:55.054292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:38:55.054370 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:38:55.054460 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:38:55.055876 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:55.056646 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:55.057856 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:38:55.058252 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:38:55.058353 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:38:55.063213 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:38:55.063230 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:38:55.063246 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:38:55.063357 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:38:55.066109 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:55.066746 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:55.066753 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:55.066755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:55.066895 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:38:55.067212 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:38:55.067215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:38:55.067217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:38:55.067639 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:38:55.067853 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 19:38:55.068132 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 19:38:55.068307 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:38:55.069305 25588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:55.160079 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 25457
I20260807 19:38:55.181023 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 25090
I20260807 19:38:55.182297 25152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:38:55.294471 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 25090
2026-08-07T19:38:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4515 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:38:55 dist-test-slave-07c2 krb5kdc[25647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:38:55 dist-test-slave-07c2 krb5kdc[25647](info): set up 2 sockets
Aug 07 19:38:55 dist-test-slave-07c2 krb5kdc[25647](info): commencing operation
krb5kdc: starting...
W20260807 19:38:56.440198 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.002s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:38:56 dist-test-slave-07c2 krb5kdc[25647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:38:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:38:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/rangeradmin_127.14.16.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/rangeradmin_127.14.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/rangerlookup_127.14.16.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/rangerlookup_127.14.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/HTTP_127.14.16.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/HTTP_127.14.16.84@KRBTEST.COM.keytab.
I20260807 19:38:56.864348 14401 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskBvS7k8/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/postgres -l logfile start
2026-08-07 19:38:59.462 UTC [25677] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 19:38:59.462 UTC [25677] LOG: listening on IPv4 address "127.14.16.84", port 59653
2026-08-07 19:38:59.466 UTC [25677] LOG: listening on Unix socket "/tmp/.s.PGSQL.59653"
2026-08-07 19:38:59.491 UTC [25679] LOG: database system was shut down at 2026-08-07 19:38:58 UTC
2026-08-07 19:38:59.495 UTC [25677] LOG: database system is ready to accept connections
I20260807 19:39:00.518975 14401 mini_postgres.cc:96] Postgres bound to 59653
2026-08-07 19:39:00.545 UTC [25689] FATAL: database "slave" does not exist
127.14.16.84:59653 - accepting connections
I20260807 19:39:00.546751 14401 mini_ranger.cc:161] Starting Ranger...
I20260807 19:39:00.581301 14401 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 19:39:01.086030 14401 mini_ranger.cc:87] Created ranger Postgres database
I20260807 19:39:01.086162 14401 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-admin
2026-08-07 19:39:01,404 [I] DB FLAVOR :POSTGRES
2026-08-07 19:39:01,405 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:39:01,405 [I] Checking connection..
2026-08-07 19:39:01,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 19:39:01,675 [I] Checking connection passed.
2026-08-07 19:39:01,675 [I] --------- Verifying version history table ---------
2026-08-07 19:39:01,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:39:01,943 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:39:01,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:39:02,215 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:39:02,216 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 19:39:02,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 19:39:02.478 UTC [25755] WARNING: there is no transaction in progress
2026-08-07 19:39:02,486 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 19:39:02,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:39:02,751 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 19:39:02,751 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:39:02,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 19:39:03,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 19:39:03,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-07c2.c.gcp-upstream.internal','N') ;"
2026-08-07 19:39:03,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:39:03,820 [I] Table x_portal_user does not exist in database ranger
2026-08-07 19:39:03,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:39:04,072 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 19:39:04,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:39:04,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:39:04,583 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 19:39:04,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 19:39:05.591 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.607 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.624 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.639 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.658 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.678 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.743 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.750 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.765 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.775 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.786 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.799 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.809 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.815 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.822 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:05.829 UTC [25867] WARNING: there is no transaction in progress
2026-08-07 19:39:06,467 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 19:39:06,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 19:39:06,718 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 19:39:06,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:39:06,971 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 19:39:06,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:39:07,242 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 19:39:07,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:39:07,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:39:07,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-07c2.c.gcp-upstream.internal';"
2026-08-07 19:39:08,017 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 19:39:08,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 19:39:08,288 [I] DB_PATCHES have already been applied
I20260807 19:39:08.294620 14401 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-admin:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskBvS7k8/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 19:39:08.294721 14401 mini_ranger.cc:193] Using host: 127.14.16.84
I20260807 19:39:08.317416 14401 mini_ranger.cc:227] Ranger admin URL: http://127.14.16.84:59425
Aug 07, 2026 7:39:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:39:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:39:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 7:39:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 7:39:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 7:39:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 7:39:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 19:39:09.519852 14401 mini_ranger.cc:160] Time spent starting Ranger: real 8.973s user 0.005s sys 0.066s
Aug 07, 2026 7:39:29 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 7:39:29 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 7:39:29 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 7:39:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 7:39:29 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 7:39:30 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 19:39:31.022389 14401 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/rangerkms_127.14.16.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/rangerkms_127.14.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.16.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.16.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.16.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/HTTP_127.14.16.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.16.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/HTTP_127.14.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 19:39:31 dist-test-slave-07c2 krb5kdc[25647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131571, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 19:39:31.255692 14401 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 19:39:31.285418 14401 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 19:39:32.105650 14401 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 19:39:32.105877 14401 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-kms
I20260807 19:39:32.106016 14401 mini_ranger_kms.cc:227] Using postgres at 127.14.16.84:59653
2026-08-07 19:39:32,566 [I] DB FLAVOR :POSTGRES
2026-08-07 19:39:32,566 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:39:32,566 [I] Checking connection
2026-08-07 19:39:32,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 19:39:32,835 [I] connection success
2026-08-07 19:39:32,836 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 19:39:32,836 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 19:39:32,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 19:39:33,077 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 19:39:33,077 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:39:33,078 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 19:39:33,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:59653/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 19:39:33,361 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 19:39:33.884287 14401 mini_ranger_kms.cc:310] Created kms service
I20260807 19:39:34.081532 14401 mini_ranger_kms.cc:326] Created kudu user
I20260807 19:39:34.129262 14401 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 19:39:34 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131571, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:39:34.508497 14401 mini_ranger_kms.cc:384] Added ranger policy
I20260807 19:39:34.509930 14401 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-kms:/tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 19:39:34.510015 14401 mini_ranger_kms.cc:240] Using host: 127.14.16.84
I20260807 19:39:34.536513 14401 mini_ranger_kms.cc:276] Ranger KMS PID: 26051
I20260807 19:39:34.536717 14401 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.16.84:52851
Aug 07, 2026 7:39:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:39:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:39:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 7:39:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 7:39:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 7:39:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 19:39:36 dist-test-slave-07c2 krb5kdc[25647](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131576, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.16.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:39:36.893707 14401 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.638s user 0.131s sys 0.377s
I20260807 19:39:36.893988 14401 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 19:39:36.894222 14401 mini_ranger_kms.cc:401] 127.14.16.84:52851/kms/v1/keys
Aug 07 19:39:37 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131576, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.16.84@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
Aug 07 19:39:37 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131576, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.16.84@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
Aug 07 19:39:39 dist-test-slave-07c2 krb5kdc[25647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:39:39.997013 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:44559
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:44559
--ranger_config_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.16.84:52851/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:39:40.364018 26103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:39:40.364367 26103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:39:40.364518 26103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:39:40.371833 26103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:39:40.371974 26103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:39:40.372071 26103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:39:40.372169 26103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:39:40.372246 26103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:39:40.382257 26103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.16.84:52851/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:44559
--ranger_config_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:44559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:39:40.383610 26103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:39:40.384907 26103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:39:40.396595 26109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:39:40.396682 26108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:39:40.396708 26111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:39:40.397717 26103 server_base.cc:1034] running on GCE node
I20260807 19:39:40.398451 26103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:39:40.399842 26103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:39:40.401065 26103 hybrid_clock.cc:648] HybridClock initialized: now 1786131580401021 us; error 47 us; skew 500 ppm
I20260807 19:39:40.401429 26103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:39:40 dist-test-slave-07c2 krb5kdc[25647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131580, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:39:40.404557 26103 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:39:40.405906 26103 webserver.cc:460] Webserver started at http://127.14.16.126:36043/ using document root <none> and password file <none>
I20260807 19:39:40.406502 26103 fs_manager.cc:359] Metadata directory not provided
I20260807 19:39:40.406600 26103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:39:40.406844 26103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:39:40 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131580, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:39:40.457391 26103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "76f292888ee84367a477f119e39bf879"
format_stamp: "Formatted at 2026-08-07 19:39:40 on dist-test-slave-07c2"
server_key: "b7b71fec844bb852d3f791b56900681d"
server_key_iv: "f78a52d6be53a68577c4f476e598df89"
server_key_version: "kuduclusterkey@0"
I20260807 19:39:40.458254 26103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "76f292888ee84367a477f119e39bf879"
format_stamp: "Formatted at 2026-08-07 19:39:40 on dist-test-slave-07c2"
server_key: "b7b71fec844bb852d3f791b56900681d"
server_key_iv: "f78a52d6be53a68577c4f476e598df89"
server_key_version: "kuduclusterkey@0"
I20260807 19:39:40.464543 26103 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 19:39:40.480515 26118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:39:40.482087 26103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:39:40.482298 26103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "76f292888ee84367a477f119e39bf879"
format_stamp: "Formatted at 2026-08-07 19:39:40 on dist-test-slave-07c2"
server_key: "b7b71fec844bb852d3f791b56900681d"
server_key_iv: "f78a52d6be53a68577c4f476e598df89"
server_key_version: "kuduclusterkey@0"
I20260807 19:39:40.482532 26103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:39:40.507449 26103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:39:40.511734 26103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:39:40.512100 26103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:39:40.532436 26103 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:44559
I20260807 19:39:40.532476 26171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:44559 every 8 connection(s)
I20260807 19:39:40.533813 26103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:39:40.538549 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:39:40.542812 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 26103
I20260807 19:39:40.543030 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:39:40.543735 14401 external_mini_cluster.cc:1428] Setting key 9d9d35c6ae619278f9ddbb9f432a4237
I20260807 19:39:40.551585 26172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879: Bootstrap starting.
I20260807 19:39:40.555120 26172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879: Neither blocks nor log segments found. Creating new log.
I20260807 19:39:40.556205 26172 log.cc:824] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879: Log is configured to *not* fsync() on all Append() calls
I20260807 19:39:40.558971 26172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879: No bootstrap required, opened a new log
Aug 07 19:39:40 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:39:40.566664 26172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f292888ee84367a477f119e39bf879" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44559 } }
I20260807 19:39:40.567050 26172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:39:40.567170 26172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f292888ee84367a477f119e39bf879, State: Initialized, Role: FOLLOWER
I20260807 19:39:40.567869 26172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f292888ee84367a477f119e39bf879" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44559 } }
I20260807 19:39:40.568166 26172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:39:40.568322 26172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:39:40.568517 26172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:39:40.571427 26172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f292888ee84367a477f119e39bf879" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44559 } }
I20260807 19:39:40.571822 26175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:40.546034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35328 (local address 127.14.16.126:44559)
0807 19:39:40.551893 (+ 5859us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:40.551909 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:40.552366 (+ 457us) server_negotiation.cc:407] Connection header received
0807 19:39:40.554297 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:40.554319 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:40.554691 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:40.555077 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:39:40.556487 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:40.557880 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:40.559133 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:40.559866 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:40.560065 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:40.564174 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:39:40.564181 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:39:40.564197 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:39:40.564238 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:39:40.567414 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:40.568346 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:40.568353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:40.568358 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:40.568563 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:40.569074 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:40.569077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:40.569079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:40.569639 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:39:40.569878 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:40.570321 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 19:39:40.570683 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5600,"thread_start_us":5319,"threads_started":1}
I20260807 19:39:40.572031 26172 leader_election.cc:302] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76f292888ee84367a477f119e39bf879; no voters:
I20260807 19:39:40.572706 26172 leader_election.cc:288] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:39:40.572853 26177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:39:40.574726 26172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:39:40.577898 26177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [term 1 LEADER]: Becoming Leader. State: Replica: 76f292888ee84367a477f119e39bf879, State: Running, Role: LEADER
I20260807 19:39:40.578636 26177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f292888ee84367a477f119e39bf879" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44559 } }
I20260807 19:39:40.582268 26179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76f292888ee84367a477f119e39bf879. Latest consensus state: current_term: 1 leader_uuid: "76f292888ee84367a477f119e39bf879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f292888ee84367a477f119e39bf879" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44559 } } }
I20260807 19:39:40.582324 26178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76f292888ee84367a477f119e39bf879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f292888ee84367a477f119e39bf879" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 44559 } } }
I20260807 19:39:40.582624 26179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [sys.catalog]: This master's current role is: LEADER
I20260807 19:39:40.582688 26178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879 [sys.catalog]: This master's current role is: LEADER
I20260807 19:39:40.583552 26189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:39:40.591144 26189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:39:40.591871 26172 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 19:39:40.618757 26189 catalog_manager.cc:1347] Generated new cluster ID: 8e1296a4d61542778eccac1a0827bcba
I20260807 19:39:40.618908 26189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:39:40.657430 26189 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:39:40.659554 26189 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:39:40.680799 26189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879: Generated new TSK 0
I20260807 19:39:40.681879 26189 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:39:41.339524 26216 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:39:41.458992 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.16.84:52851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44559
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:39:41 dist-test-slave-07c2 krb5kdc[25647](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131581, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 19:39:41.947260 26219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:39:41.947736 26219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:39:41.947898 26219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:39:41.959975 26219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:39:41.960152 26219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:39:41.960353 26219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:39:41.976392 26219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.16.84:52851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:39:41.978070 26219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:39:41.979795 26219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:39:41.998374 26230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:39:42.003017 26232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:39:42.003577 26219 server_base.cc:1034] running on GCE node
W20260807 19:39:42.004048 26229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:39:42.004972 26219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:39:42.007614 26219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:39:42.008932 26219 hybrid_clock.cc:648] HybridClock initialized: now 1786131582008820 us; error 103 us; skew 500 ppm
I20260807 19:39:42.009397 26219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:39:42 dist-test-slave-07c2 krb5kdc[25647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131582, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:39:42.013981 26219 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:39:42.020093 26219 webserver.cc:460] Webserver started at http://127.14.16.65:39053/ using document root <none> and password file <none>
I20260807 19:39:42.021068 26219 fs_manager.cc:359] Metadata directory not provided
I20260807 19:39:42.021229 26219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:39:42.021622 26219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:39:42 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131582, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:39:42.044647 26219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "1b6f0201f22d4eb48bd5b0cef737e383"
format_stamp: "Formatted at 2026-08-07 19:39:42 on dist-test-slave-07c2"
server_key: "043411e283c8170118ae5c447d07d39c"
server_key_iv: "52fe7165576d7f8454ff08f73efc2e43"
server_key_version: "kuduclusterkey@0"
I20260807 19:39:42.045893 26219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "1b6f0201f22d4eb48bd5b0cef737e383"
format_stamp: "Formatted at 2026-08-07 19:39:42 on dist-test-slave-07c2"
server_key: "043411e283c8170118ae5c447d07d39c"
server_key_iv: "52fe7165576d7f8454ff08f73efc2e43"
server_key_version: "kuduclusterkey@0"
I20260807 19:39:42.067380 26219 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.012s sys 0.008s
I20260807 19:39:42.100111 26240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:39:42.105582 26219 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.008s sys 0.008s
I20260807 19:39:42.105916 26219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "1b6f0201f22d4eb48bd5b0cef737e383"
format_stamp: "Formatted at 2026-08-07 19:39:42 on dist-test-slave-07c2"
server_key: "043411e283c8170118ae5c447d07d39c"
server_key_iv: "52fe7165576d7f8454ff08f73efc2e43"
server_key_version: "kuduclusterkey@0"
I20260807 19:39:42.106261 26219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:39:42.131681 26219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:39:42.136240 26219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:39:42.136588 26219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:39:42.137729 26219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:39:42.139890 26219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:39:42.140020 26219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 19:39:42.140262 26219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:39:42.140355 26219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:39:42 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131581, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:39:42.380178 26219 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:39807
I20260807 19:39:42.380244 26358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:39807 every 8 connection(s)
I20260807 19:39:42.382081 26219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:39:42.388136 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 26219
I20260807 19:39:42.388367 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:39:42.389210 14401 external_mini_cluster.cc:1428] Setting key 2e1e3bc8a9e23d2b3284766e572df9b6
Aug 07 19:39:42 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131582, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
I20260807 19:39:42.444301 26361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:42.392079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:34005 (local address 127.14.16.126:44559)
0807 19:39:42.415068 (+ 22989us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:42.415078 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:42.422030 (+ 6952us) server_negotiation.cc:407] Connection header received
0807 19:39:42.424792 (+ 2762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:42.424799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:42.424905 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:42.424988 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:39:42.427258 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:42.428448 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:42.430869 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:42.431554 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:42.431651 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:42.436369 (+ 4718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:39:42.436376 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:39:42.436381 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:39:42.436434 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:39:42.440206 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:42.441225 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:42.441231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:42.441235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:42.441401 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:42.442004 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:42.442009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:42.442013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:42.442410 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:39:42.442542 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:42.443641 (+ 1099us) server_negotiation.cc:299] Negotiation successful
0807 19:39:42.443899 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22806,"thread_start_us":22671,"threads_started":1}
I20260807 19:39:42.446627 26362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:42.392809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44559 (local address 127.14.16.65:34005)
0807 19:39:42.421820 (+ 29011us) negotiation.cc:107] Waiting for socket to connect
0807 19:39:42.421881 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 19:39:42.424447 (+ 2566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:39:42.425557 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:39:42.425568 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:39:42.426231 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:39:42.427030 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:42.427058 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:42.428692 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:42.428698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:42.430694 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:42.430709 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:42.432023 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:42.432029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:42.432347 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:42.433195 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:39:42.433225 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:39:42.436088 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:39:42.440401 (+ 4313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:39:42.440411 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:39:42.440431 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:39:42.441047 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:39:42.441569 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:39:42.441574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:39:42.441577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:39:42.441855 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:39:42.442661 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:39:42.442670 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:39:42.443129 (+ 459us) client_negotiation.cc:769] Sending connection context
0807 19:39:42.444532 (+ 1403us) client_negotiation.cc:240] Negotiation successful
0807 19:39:42.445149 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28665,"thread_start_us":28438,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
I20260807 19:39:42.448786 26359 heartbeater.cc:347] Connected to a master server at 127.14.16.126:44559
I20260807 19:39:42.449106 26359 heartbeater.cc:464] Registering TS with master...
I20260807 19:39:42.450033 26359 heartbeater.cc:511] Master 127.14.16.126:44559 requested a full tablet report, sending...
I20260807 19:39:42.452919 26136 ts_manager.cc:194] Registered new tserver with Master: 1b6f0201f22d4eb48bd5b0cef737e383 (127.14.16.65:39807)
I20260807 19:39:42.455006 26136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:34005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:39:42.490986 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.16.84:52851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44559
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:39:42 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131581, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
W20260807 19:39:43.013088 26366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:39:43.013422 26366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:39:43.013624 26366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:39:43.022203 26366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:39:43.022341 26366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:39:43.022573 26366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:39:43.033492 26366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.16.84:52851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:39:43.034962 26366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:39:43.036389 26366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:39:43.052711 26380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:39:43.052703 26381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:39:43.052703 26383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:39:43.053646 26366 server_base.cc:1034] running on GCE node
I20260807 19:39:43.054293 26366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:39:43.055387 26366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:39:43.056655 26366 hybrid_clock.cc:648] HybridClock initialized: now 1786131583056605 us; error 46 us; skew 500 ppm
I20260807 19:39:43.056993 26366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:39:43 dist-test-slave-07c2 krb5kdc[25647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131583, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:39:43.060431 26366 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:39:43.062096 26366 webserver.cc:460] Webserver started at http://127.14.16.66:46299/ using document root <none> and password file <none>
I20260807 19:39:43.062798 26366 fs_manager.cc:359] Metadata directory not provided
I20260807 19:39:43.062927 26366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:39:43.063258 26366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:39:43 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131583, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:39:43.082597 26366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "96d9ad21abc94ab7b7ab89303cda594c"
format_stamp: "Formatted at 2026-08-07 19:39:43 on dist-test-slave-07c2"
server_key: "dc48dd0d169d71c66ee59944683cf137"
server_key_iv: "0212a07b913312b46b72cf767b874656"
server_key_version: "kuduclusterkey@0"
I20260807 19:39:43.084007 26366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "96d9ad21abc94ab7b7ab89303cda594c"
format_stamp: "Formatted at 2026-08-07 19:39:43 on dist-test-slave-07c2"
server_key: "dc48dd0d169d71c66ee59944683cf137"
server_key_iv: "0212a07b913312b46b72cf767b874656"
server_key_version: "kuduclusterkey@0"
I20260807 19:39:43.091629 26366 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260807 19:39:43.107524 26390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:39:43.109167 26366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:39:43.109351 26366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "96d9ad21abc94ab7b7ab89303cda594c"
format_stamp: "Formatted at 2026-08-07 19:39:43 on dist-test-slave-07c2"
server_key: "dc48dd0d169d71c66ee59944683cf137"
server_key_iv: "0212a07b913312b46b72cf767b874656"
server_key_version: "kuduclusterkey@0"
I20260807 19:39:43.109606 26366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:39:43.128358 26366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:39:43.132611 26366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:39:43.132925 26366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:39:43.134004 26366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:39:43.135484 26366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:39:43.135556 26366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:39:43.135687 26366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:39:43.135736 26366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:39:43.173112 26366 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:44343
I20260807 19:39:43.173138 26506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:44343 every 8 connection(s)
I20260807 19:39:43.174422 26366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:39:43.183276 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 26366
I20260807 19:39:43.183565 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:39:43.184299 14401 external_mini_cluster.cc:1428] Setting key f662f7273cb75bec44cfb36e4216db1d
Aug 07 19:39:43 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131583, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:39:43.197875 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.177394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:47879 (local address 127.14.16.126:44559)
0807 19:39:43.177899 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:43.177905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:43.179045 (+ 1140us) server_negotiation.cc:407] Connection header received
0807 19:39:43.181759 (+ 2714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:43.181764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:43.181837 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:43.181936 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:39:43.184071 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.185200 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.187535 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.187940 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.188086 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.191761 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:39:43.191766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:39:43.191769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:39:43.191802 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:39:43.194529 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:43.195376 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:43.195380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:43.195382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:43.195508 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:43.196005 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:43.196009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:43.196011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:43.196300 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:39:43.196482 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:43.197214 (+ 732us) server_negotiation.cc:299] Negotiation successful
0807 19:39:43.197460 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":244,"threads_started":1}
I20260807 19:39:43.199142 26510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.177997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44559 (local address 127.14.16.66:47879)
0807 19:39:43.178828 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0807 19:39:43.178894 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 19:39:43.181400 (+ 2506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:39:43.182252 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:39:43.182265 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:39:43.182934 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:39:43.183831 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.183857 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.185409 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.185418 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.187381 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.187394 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.188183 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.188191 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.188444 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.189173 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:39:43.189196 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:39:43.191516 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:39:43.194718 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:39:43.194727 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:39:43.194751 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:39:43.195209 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:39:43.195636 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:39:43.195641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:39:43.195645 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:39:43.195868 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:39:43.196439 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:39:43.196447 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:39:43.196832 (+ 385us) client_negotiation.cc:769] Sending connection context
0807 19:39:43.197108 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 19:39:43.197691 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":330,"threads_started":1}
I20260807 19:39:43.200806 26507 heartbeater.cc:347] Connected to a master server at 127.14.16.126:44559
I20260807 19:39:43.201149 26507 heartbeater.cc:464] Registering TS with master...
I20260807 19:39:43.201881 26507 heartbeater.cc:511] Master 127.14.16.126:44559 requested a full tablet report, sending...
I20260807 19:39:43.203588 26136 ts_manager.cc:194] Registered new tserver with Master: 96d9ad21abc94ab7b7ab89303cda594c (127.14.16.66:44343)
I20260807 19:39:43.204432 26136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:47879
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:39:43.261327 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.16.84:52851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44559
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:34847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:39:43.459002 26359 heartbeater.cc:503] Master 127.14.16.126:44559 was elected leader, sending a full tablet report...
W20260807 19:39:43.633038 26513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:39:43.633332 26513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:39:43.633508 26513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:39:43.640952 26513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:39:43.641064 26513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:39:43.641278 26513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:39:43.651234 26513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:34847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.16.84:52851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:44559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:39:43.652606 26513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:39:43.653921 26513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:39:43.667716 26518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:39:43.667635 26521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:39:43.667670 26519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:39:43.668557 26513 server_base.cc:1034] running on GCE node
I20260807 19:39:43.669225 26513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:39:43.670225 26513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:39:43.671396 26513 hybrid_clock.cc:648] HybridClock initialized: now 1786131583671356 us; error 34 us; skew 500 ppm
I20260807 19:39:43.671716 26513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:39:43 dist-test-slave-07c2 krb5kdc[25647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131583, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:39:43.675001 26513 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:39:43.676614 26513 webserver.cc:460] Webserver started at http://127.14.16.67:39277/ using document root <none> and password file <none>
I20260807 19:39:43.677313 26513 fs_manager.cc:359] Metadata directory not provided
I20260807 19:39:43.677464 26513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:39:43.677783 26513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:39:43 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131583, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:39:43.694621 26513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "ed446a66dad3435aa6207e96850eec17"
format_stamp: "Formatted at 2026-08-07 19:39:43 on dist-test-slave-07c2"
server_key: "8c1e5f513db6e5eede559bfd8b78fb03"
server_key_iv: "4627f7318e8a381dea8fb5c063e2e9d6"
server_key_version: "kuduclusterkey@0"
I20260807 19:39:43.695602 26513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "ed446a66dad3435aa6207e96850eec17"
format_stamp: "Formatted at 2026-08-07 19:39:43 on dist-test-slave-07c2"
server_key: "8c1e5f513db6e5eede559bfd8b78fb03"
server_key_iv: "4627f7318e8a381dea8fb5c063e2e9d6"
server_key_version: "kuduclusterkey@0"
I20260807 19:39:43.701423 26513 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:39:43.718335 26528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:39:43.719825 26513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:39:43.720010 26513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "ed446a66dad3435aa6207e96850eec17"
format_stamp: "Formatted at 2026-08-07 19:39:43 on dist-test-slave-07c2"
server_key: "8c1e5f513db6e5eede559bfd8b78fb03"
server_key_iv: "4627f7318e8a381dea8fb5c063e2e9d6"
server_key_version: "kuduclusterkey@0"
I20260807 19:39:43.720255 26513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:39:43.746304 26513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:39:43.750406 26513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:39:43.750751 26513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:39:43.751751 26513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:39:43.753126 26513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:39:43.753211 26513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:39:43.753371 26513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:39:43.753422 26513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:39:43.786339 26513 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:39455
I20260807 19:39:43.786329 26645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:39455 every 8 connection(s)
I20260807 19:39:43.787760 26513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:39:43.795475 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 26513
I20260807 19:39:43.795680 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:39:43.796108 14401 external_mini_cluster.cc:1428] Setting key a634757b179ccfc4f47fb1d7a152d129
Aug 07 19:39:43 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131583, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:39:43.807571 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.790374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:50507 (local address 127.14.16.126:44559)
0807 19:39:43.790986 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:43.790991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:43.791747 (+ 756us) server_negotiation.cc:407] Connection header received
0807 19:39:43.793497 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:43.793503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:43.793589 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:43.793709 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:39:43.795086 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.795961 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.797773 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.798336 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.798472 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.801886 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:39:43.801891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:39:43.801894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:39:43.801925 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:39:43.804556 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:43.805284 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:43.805293 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:43.805297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:43.805432 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:43.805887 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:43.805892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:43.805897 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:43.806156 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:39:43.806330 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:43.806968 (+ 638us) server_negotiation.cc:299] Negotiation successful
0807 19:39:43.807180 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":321,"threads_started":1}
I20260807 19:39:43.808585 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.790828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:44559 (local address 127.14.16.67:50507)
0807 19:39:43.791566 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0807 19:39:43.791601 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 19:39:43.793221 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:39:43.793940 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:39:43.793948 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:39:43.794380 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:39:43.794909 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.794931 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.796112 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.796118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.797626 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.797638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.798556 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.798565 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.798823 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.799561 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:39:43.799584 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:39:43.801648 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:39:43.804701 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:39:43.804709 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:39:43.804725 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:39:43.805140 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:39:43.805547 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:39:43.805552 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:39:43.805556 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:39:43.805763 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:39:43.806293 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:39:43.806300 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:39:43.806614 (+ 314us) client_negotiation.cc:769] Sending connection context
0807 19:39:43.806877 (+ 263us) client_negotiation.cc:240] Negotiation successful
0807 19:39:43.807410 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":284,"threads_started":1}
I20260807 19:39:43.810065 26646 heartbeater.cc:347] Connected to a master server at 127.14.16.126:44559
I20260807 19:39:43.810403 26646 heartbeater.cc:464] Registering TS with master...
I20260807 19:39:43.811324 26646 heartbeater.cc:511] Master 127.14.16.126:44559 requested a full tablet report, sending...
I20260807 19:39:43.812783 26136 ts_manager.cc:194] Registered new tserver with Master: ed446a66dad3435aa6207e96850eec17 (127.14.16.67:39455)
I20260807 19:39:43.813608 26136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:50507
I20260807 19:39:43.823963 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:39:43.840826 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.828679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35334 (local address 127.14.16.126:44559)
0807 19:39:43.828872 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:43.828876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:43.829384 (+ 508us) server_negotiation.cc:407] Connection header received
0807 19:39:43.829532 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:43.829535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:43.829612 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:43.829694 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:39:43.831077 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.831857 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.833122 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.833544 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.833637 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.836025 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:39:43.836030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:39:43.836033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:39:43.836061 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:39:43.838481 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:43.839138 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:43.839141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:43.839143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:43.839262 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:43.839631 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:43.839634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:43.839636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:43.839914 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:39:43.840090 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:43.840408 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 19:39:43.840581 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 19:39:43.845014 26136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:39:43.852212 26136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:39:43.877521 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.864978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33566 (local address 127.14.16.66:44343)
0807 19:39:43.867196 (+ 2218us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:43.867207 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:43.868635 (+ 1428us) server_negotiation.cc:407] Connection header received
0807 19:39:43.868932 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:43.868939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:43.869179 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:43.869365 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:39:43.870290 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.871756 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.874627 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.876063 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.876193 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.876392 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:43.876998 (+ 606us) server_negotiation.cc:299] Negotiation successful
0807 19:39:43.877179 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1995,"thread_start_us":2231,"threads_started":1}
I20260807 19:39:43.877671 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.866687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39807 (local address 127.0.0.1:52276)
0807 19:39:43.869672 (+ 2985us) negotiation.cc:107] Waiting for socket to connect
0807 19:39:43.869688 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:39:43.869812 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:39:43.870415 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:39:43.870422 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:39:43.870447 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:39:43.870836 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.870848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.872664 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.872674 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.874239 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.874262 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.876263 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.876270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.876633 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.876778 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 19:39:43.876920 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 19:39:43.877124 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2818,"mutex_wait_us":109,"thread_start_us":2662,"threads_started":1}
I20260807 19:39:43.877833 26659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.864605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:44343 (local address 127.0.0.1:33566)
0807 19:39:43.868506 (+ 3901us) negotiation.cc:107] Waiting for socket to connect
0807 19:39:43.868527 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:39:43.868740 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:39:43.869682 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:39:43.869688 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:39:43.869720 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:39:43.870112 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.870120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.871920 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.871928 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.874472 (+ 2544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.874485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.876331 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.876337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.876635 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.876778 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 19:39:43.876918 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 19:39:43.877129 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3771,"mutex_wait_us":232,"spinlock_wait_cycles":13184,"thread_start_us":1022,"threads_started":1}
I20260807 19:39:43.878415 26664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.867036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52276 (local address 127.14.16.65:39807)
0807 19:39:43.867969 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:43.867981 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:43.869803 (+ 1822us) server_negotiation.cc:407] Connection header received
0807 19:39:43.869989 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:43.869999 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:43.870215 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:43.870417 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:39:43.871000 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.872501 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.874400 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.875959 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.876123 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.876428 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:43.877878 (+ 1450us) server_negotiation.cc:299] Negotiation successful
0807 19:39:43.878034 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":533,"threads_started":1}
I20260807 19:39:43.879251 26660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.864464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:39455 (local address 127.0.0.1:59738)
0807 19:39:43.866021 (+ 1557us) negotiation.cc:107] Waiting for socket to connect
0807 19:39:43.866063 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:39:43.866240 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:39:43.869000 (+ 2760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:39:43.869011 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:39:43.869051 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:39:43.869660 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.869672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.871605 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.871610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.874470 (+ 2860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.874485 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.878435 (+ 3950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.878442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.878616 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.878648 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:39:43.878729 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 19:39:43.878851 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1311,"mutex_wait_us":95,"thread_start_us":3763,"threads_started":1}
I20260807 19:39:43.879946 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.864755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59738 (local address 127.14.16.67:39455)
0807 19:39:43.865668 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:43.865679 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:43.867865 (+ 2186us) server_negotiation.cc:407] Connection header received
0807 19:39:43.868021 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:43.868033 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:43.868337 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:43.868596 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:39:43.869930 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.871466 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.876514 (+ 5048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.878010 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.878365 (+ 355us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.879385 (+ 1020us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:43.879460 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 19:39:43.879611 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":479,"threads_started":1}
I20260807 19:39:43.882179 26439 tablet_service.cc:1467] Processing CreateTablet for tablet fb59613824ea455ab6ed47473a2aa2cf (DEFAULT_TABLE table=test-table [id=b86cecb820594b9d812cb61b02f204c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:39:43.882838 26290 tablet_service.cc:1467] Processing CreateTablet for tablet fb59613824ea455ab6ed47473a2aa2cf (DEFAULT_TABLE table=test-table [id=b86cecb820594b9d812cb61b02f204c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:39:43.883112 26578 tablet_service.cc:1467] Processing CreateTablet for tablet fb59613824ea455ab6ed47473a2aa2cf (DEFAULT_TABLE table=test-table [id=b86cecb820594b9d812cb61b02f204c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:39:43.883738 26439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb59613824ea455ab6ed47473a2aa2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:39:43.884279 26290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb59613824ea455ab6ed47473a2aa2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:39:43.884451 26578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb59613824ea455ab6ed47473a2aa2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:39:43.895804 26665 tablet_bootstrap.cc:492] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17: Bootstrap starting.
I20260807 19:39:43.899895 26665 tablet_bootstrap.cc:654] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17: Neither blocks nor log segments found. Creating new log.
I20260807 19:39:43.900995 26665 log.cc:824] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17: Log is configured to *not* fsync() on all Append() calls
I20260807 19:39:43.902410 26666 tablet_bootstrap.cc:492] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383: Bootstrap starting.
I20260807 19:39:43.903205 26667 tablet_bootstrap.cc:492] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c: Bootstrap starting.
I20260807 19:39:43.904316 26665 tablet_bootstrap.cc:492] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17: No bootstrap required, opened a new log
I20260807 19:39:43.904600 26665 ts_tablet_manager.cc:1397] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260807 19:39:43.906579 26666 tablet_bootstrap.cc:654] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383: Neither blocks nor log segments found. Creating new log.
I20260807 19:39:43.907407 26667 tablet_bootstrap.cc:654] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c: Neither blocks nor log segments found. Creating new log.
I20260807 19:39:43.907974 26666 log.cc:824] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383: Log is configured to *not* fsync() on all Append() calls
I20260807 19:39:43.908747 26667 log.cc:824] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:39:43.911067 26666 tablet_bootstrap.cc:492] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383: No bootstrap required, opened a new log
I20260807 19:39:43.911396 26666 ts_tablet_manager.cc:1397] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:39:43.913705 26667 tablet_bootstrap.cc:492] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c: No bootstrap required, opened a new log
I20260807 19:39:43.914091 26667 ts_tablet_manager.cc:1397] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260807 19:39:43.914021 26665 raft_consensus.cc:348] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed446a66dad3435aa6207e96850eec17" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 39455 } } peers { permanent_uuid: "96d9ad21abc94ab7b7ab89303cda594c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44343 } } peers { permanent_uuid: "1b6f0201f22d4eb48bd5b0cef737e383" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39807 } }
I20260807 19:39:43.914458 26665 raft_consensus.cc:374] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:39:43.914564 26665 raft_consensus.cc:729] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed446a66dad3435aa6207e96850eec17, State: Initialized, Role: FOLLOWER
I20260807 19:39:43.915170 26665 consensus_queue.cc:260] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed446a66dad3435aa6207e96850eec17" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 39455 } } peers { permanent_uuid: "96d9ad21abc94ab7b7ab89303cda594c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44343 } } peers { permanent_uuid: "1b6f0201f22d4eb48bd5b0cef737e383" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39807 } }
I20260807 19:39:43.916347 26646 heartbeater.cc:503] Master 127.14.16.126:44559 was elected leader, sending a full tablet report...
I20260807 19:39:43.916996 26665 ts_tablet_manager.cc:1428] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260807 19:39:43.921936 26666 raft_consensus.cc:348] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed446a66dad3435aa6207e96850eec17" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 39455 } } peers { permanent_uuid: "96d9ad21abc94ab7b7ab89303cda594c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44343 } } peers { permanent_uuid: "1b6f0201f22d4eb48bd5b0cef737e383" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39807 } }
I20260807 19:39:43.922372 26666 raft_consensus.cc:374] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:39:43.922478 26666 raft_consensus.cc:729] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6f0201f22d4eb48bd5b0cef737e383, State: Initialized, Role: FOLLOWER
I20260807 19:39:43.923079 26666 consensus_queue.cc:260] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed446a66dad3435aa6207e96850eec17" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 39455 } } peers { permanent_uuid: "96d9ad21abc94ab7b7ab89303cda594c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44343 } } peers { permanent_uuid: "1b6f0201f22d4eb48bd5b0cef737e383" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39807 } }
I20260807 19:39:43.924919 26666 ts_tablet_manager.cc:1428] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260807 19:39:43.925339 26667 raft_consensus.cc:348] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed446a66dad3435aa6207e96850eec17" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 39455 } } peers { permanent_uuid: "96d9ad21abc94ab7b7ab89303cda594c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44343 } } peers { permanent_uuid: "1b6f0201f22d4eb48bd5b0cef737e383" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39807 } }
I20260807 19:39:43.925752 26667 raft_consensus.cc:374] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:39:43.925865 26667 raft_consensus.cc:729] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96d9ad21abc94ab7b7ab89303cda594c, State: Initialized, Role: FOLLOWER
I20260807 19:39:43.926476 26667 consensus_queue.cc:260] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed446a66dad3435aa6207e96850eec17" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 39455 } } peers { permanent_uuid: "96d9ad21abc94ab7b7ab89303cda594c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44343 } } peers { permanent_uuid: "1b6f0201f22d4eb48bd5b0cef737e383" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39807 } }
I20260807 19:39:43.927604 26507 heartbeater.cc:503] Master 127.14.16.126:44559 was elected leader, sending a full tablet report...
I20260807 19:39:43.928131 26667 ts_tablet_manager.cc:1428] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260807 19:39:43.989269 26671 raft_consensus.cc:482] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:39:43.989619 26671 raft_consensus.cc:504] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed446a66dad3435aa6207e96850eec17" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 39455 } } peers { permanent_uuid: "96d9ad21abc94ab7b7ab89303cda594c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44343 } } peers { permanent_uuid: "1b6f0201f22d4eb48bd5b0cef737e383" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39807 } }
I20260807 19:39:43.991740 26671 leader_election.cc:288] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96d9ad21abc94ab7b7ab89303cda594c (127.14.16.66:44343), 1b6f0201f22d4eb48bd5b0cef737e383 (127.14.16.65:39807)
I20260807 19:39:43.998308 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.992171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:45585 (local address 127.14.16.66:44343)
0807 19:39:43.992385 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:43.992391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:43.992415 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:39:43.992622 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:43.992629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:43.992713 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:43.992808 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:39:43.993367 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.994390 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.996299 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.997349 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.997464 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.997515 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:43.997905 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 19:39:43.998047 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:39:43.998323 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.992136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:44343 (local address 127.14.16.67:45585)
0807 19:39:43.992298 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 19:39:43.992323 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:39:43.992437 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:39:43.992941 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:39:43.992947 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:39:43.992977 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:39:43.993188 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.993201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.994527 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.994533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.996128 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.996140 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.997518 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.997524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.997707 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.997765 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 19:39:43.997873 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 19:39:43.998028 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":3}
I20260807 19:39:43.998919 26664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.992269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:57009 (local address 127.14.16.65:39807)
0807 19:39:43.992458 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:43.992470 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:43.993132 (+ 662us) server_negotiation.cc:407] Connection header received
0807 19:39:43.993265 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:43.993272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:43.993359 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:43.993426 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:39:43.994056 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.995001 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.996947 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.997929 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.998102 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.998173 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:43.998599 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 19:39:43.998743 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:39:43.999670 26674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:43.992156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39807 (local address 127.14.16.67:57009)
0807 19:39:43.993026 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0807 19:39:43.993040 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:39:43.993147 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:39:43.993488 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:39:43.993494 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:39:43.993520 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:39:43.993907 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.993918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.995172 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.995179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.996797 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:39:43.996811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:39:43.998182 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:39:43.998189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:39:43.998372 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:43.998405 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:39:43.999110 (+ 705us) client_negotiation.cc:240] Negotiation successful
0807 19:39:43.999254 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"mutex_wait_us":198,"thread_start_us":369,"threads_started":1}
I20260807 19:39:43.999569 26462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb59613824ea455ab6ed47473a2aa2cf" candidate_uuid: "ed446a66dad3435aa6207e96850eec17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96d9ad21abc94ab7b7ab89303cda594c" is_pre_election: true
I20260807 19:39:44.000170 26462 raft_consensus.cc:2454] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed446a66dad3435aa6207e96850eec17 in term 0.
I20260807 19:39:44.000828 26311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb59613824ea455ab6ed47473a2aa2cf" candidate_uuid: "ed446a66dad3435aa6207e96850eec17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6f0201f22d4eb48bd5b0cef737e383" is_pre_election: true
I20260807 19:39:44.000957 26531 leader_election.cc:302] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96d9ad21abc94ab7b7ab89303cda594c, ed446a66dad3435aa6207e96850eec17; no voters:
I20260807 19:39:44.001397 26671 raft_consensus.cc:2781] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:39:44.001425 26311 raft_consensus.cc:2454] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed446a66dad3435aa6207e96850eec17 in term 0.
I20260807 19:39:44.001547 26671 raft_consensus.cc:482] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:39:44.001667 26671 raft_consensus.cc:3037] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:39:44.004051 26671 raft_consensus.cc:504] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed446a66dad3435aa6207e96850eec17" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 39455 } } peers { permanent_uuid: "96d9ad21abc94ab7b7ab89303cda594c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44343 } } peers { permanent_uuid: "1b6f0201f22d4eb48bd5b0cef737e383" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39807 } }
I20260807 19:39:44.004632 26671 leader_election.cc:288] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [CANDIDATE]: Term 1 election: Requested vote from peers 96d9ad21abc94ab7b7ab89303cda594c (127.14.16.66:44343), 1b6f0201f22d4eb48bd5b0cef737e383 (127.14.16.65:39807)
I20260807 19:39:44.005129 26311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb59613824ea455ab6ed47473a2aa2cf" candidate_uuid: "ed446a66dad3435aa6207e96850eec17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b6f0201f22d4eb48bd5b0cef737e383"
I20260807 19:39:44.005129 26462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb59613824ea455ab6ed47473a2aa2cf" candidate_uuid: "ed446a66dad3435aa6207e96850eec17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96d9ad21abc94ab7b7ab89303cda594c"
I20260807 19:39:44.005339 26462 raft_consensus.cc:3037] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:39:44.005338 26311 raft_consensus.cc:3037] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:39:44.008687 26462 raft_consensus.cc:2454] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed446a66dad3435aa6207e96850eec17 in term 1.
I20260807 19:39:44.008687 26311 raft_consensus.cc:2454] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed446a66dad3435aa6207e96850eec17 in term 1.
I20260807 19:39:44.009125 26531 leader_election.cc:302] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96d9ad21abc94ab7b7ab89303cda594c, ed446a66dad3435aa6207e96850eec17; no voters:
I20260807 19:39:44.009459 26671 raft_consensus.cc:2781] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:39:44.009915 26671 raft_consensus.cc:686] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 1 LEADER]: Becoming Leader. State: Replica: ed446a66dad3435aa6207e96850eec17, State: Running, Role: LEADER
I20260807 19:39:44.010485 26671 consensus_queue.cc:237] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed446a66dad3435aa6207e96850eec17" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 39455 } } peers { permanent_uuid: "96d9ad21abc94ab7b7ab89303cda594c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44343 } } peers { permanent_uuid: "1b6f0201f22d4eb48bd5b0cef737e383" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39807 } }
I20260807 19:39:44.015199 26136 catalog_manager.cc:5495] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed446a66dad3435aa6207e96850eec17 (127.14.16.67). New cstate: current_term: 1 leader_uuid: "ed446a66dad3435aa6207e96850eec17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed446a66dad3435aa6207e96850eec17" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 39455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96d9ad21abc94ab7b7ab89303cda594c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f0201f22d4eb48bd5b0cef737e383" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39807 } health_report { overall_health: UNKNOWN } } }
I20260807 19:39:44.033207 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:44.028018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.14.16.67:39455)
0807 19:39:44.028191 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:44.028195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:44.028242 (+ 47us) server_negotiation.cc:407] Connection header received
0807 19:39:44.028454 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:44.028457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:44.028533 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:44.028689 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:39:44.029095 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:44.029796 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:44.031242 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:44.031698 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:44.031795 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:44.032261 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:39:44.032740 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:39:44.032856 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:44.032944 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 19:39:44.033010 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 19:39:44.039383 26647 tablet.cc:2369] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:39:44.041157 26462 raft_consensus.cc:1261] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c [term 1 FOLLOWER]: Refusing update from remote peer ed446a66dad3435aa6207e96850eec17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:39:44.041153 26311 raft_consensus.cc:1261] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383 [term 1 FOLLOWER]: Refusing update from remote peer ed446a66dad3435aa6207e96850eec17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:39:44.042121 26671 consensus_queue.cc:1035] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96d9ad21abc94ab7b7ab89303cda594c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:39:44.042418 26675 consensus_queue.cc:1035] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f0201f22d4eb48bd5b0cef737e383" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:39:44.054965 26678 mvcc.cc:205] Tried to move back new op lower bound from 7315994968222564352 to 7315994968111173632. Current Snapshot: MvccSnapshot[applied={T|T < 7315994968222564352}]
I20260807 19:39:44.057058 26680 mvcc.cc:205] Tried to move back new op lower bound from 7315994968222564352 to 7315994968111173632. Current Snapshot: MvccSnapshot[applied={T|T < 7315994968222564352}]
I20260807 19:39:44.070683 26136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:39:44.070964 26136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:39:44.074826 26136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76f292888ee84367a477f119e39bf879: Sending DeleteTablet for 3 replicas of tablet fb59613824ea455ab6ed47473a2aa2cf
I20260807 19:39:44.075970 26578 tablet_service.cc:1514] Processing DeleteTablet for tablet fb59613824ea455ab6ed47473a2aa2cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:39:44 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:59738
I20260807 19:39:44.076083 26439 tablet_service.cc:1514] Processing DeleteTablet for tablet fb59613824ea455ab6ed47473a2aa2cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:39:44 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:33566
I20260807 19:39:44.076329 26290 tablet_service.cc:1514] Processing DeleteTablet for tablet fb59613824ea455ab6ed47473a2aa2cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:39:44 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:52276
I20260807 19:39:44.076915 26687 tablet_replica.cc:330] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17: stopping tablet replica
I20260807 19:39:44.077792 26688 tablet_replica.cc:330] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c: stopping tablet replica
I20260807 19:39:44.078203 26687 raft_consensus.cc:2229] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:39:44.078420 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 26219
I20260807 19:39:44.078908 26688 raft_consensus.cc:2229] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:39:44.078923 26687 raft_consensus.cc:2258] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:39:44.079406 26688 raft_consensus.cc:2258] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:39:44.081843 26688 ts_tablet_manager.cc:1905] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:39:44.084561 26687 ts_tablet_manager.cc:1905] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:39:44.086174 26689 tablet_replica.cc:330] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383: stopping tablet replica
I20260807 19:39:44.086620 26689 raft_consensus.cc:2229] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:39:44.087195 26689 raft_consensus.cc:2258] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:39:44.089505 26688 ts_tablet_manager.cc:1918] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:39:44.089684 26688 log.cc:1196] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/fb59613824ea455ab6ed47473a2aa2cf
I20260807 19:39:44.090214 26688 ts_tablet_manager.cc:1939] T fb59613824ea455ab6ed47473a2aa2cf P 96d9ad21abc94ab7b7ab89303cda594c: Deleting consensus metadata
I20260807 19:39:44.092072 26689 ts_tablet_manager.cc:1905] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:39:44.092134 26122 catalog_manager.cc:4841] TS 96d9ad21abc94ab7b7ab89303cda594c (127.14.16.66:44343): tablet fb59613824ea455ab6ed47473a2aa2cf (table test-table [id=b86cecb820594b9d812cb61b02f204c3]) successfully deleted
I20260807 19:39:44.093286 26687 ts_tablet_manager.cc:1918] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:39:44.093454 26687 log.cc:1196] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/fb59613824ea455ab6ed47473a2aa2cf
I20260807 19:39:44.094013 26687 ts_tablet_manager.cc:1939] T fb59613824ea455ab6ed47473a2aa2cf P ed446a66dad3435aa6207e96850eec17: Deleting consensus metadata
I20260807 19:39:44.095657 26123 catalog_manager.cc:4841] TS ed446a66dad3435aa6207e96850eec17 (127.14.16.67:39455): tablet fb59613824ea455ab6ed47473a2aa2cf (table test-table [id=b86cecb820594b9d812cb61b02f204c3]) successfully deleted
I20260807 19:39:44.099409 26689 ts_tablet_manager.cc:1918] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:39:44.099587 26689 log.cc:1196] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/fb59613824ea455ab6ed47473a2aa2cf
I20260807 19:39:44.100163 26689 ts_tablet_manager.cc:1939] T fb59613824ea455ab6ed47473a2aa2cf P 1b6f0201f22d4eb48bd5b0cef737e383: Deleting consensus metadata
I20260807 19:39:44.101969 26119 catalog_manager.cc:4841] TS 1b6f0201f22d4eb48bd5b0cef737e383 (127.14.16.65:39807): tablet fb59613824ea455ab6ed47473a2aa2cf (table test-table [id=b86cecb820594b9d812cb61b02f204c3]) successfully deleted
Aug 07 19:39:44 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:39:44.113986 26664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:44.081841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52278 (local address 127.14.16.65:39807)
0807 19:39:44.082095 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:44.082104 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:44.096064 (+ 13960us) server_negotiation.cc:407] Connection header received
0807 19:39:44.096292 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:44.096307 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:44.096403 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:44.096520 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:39:44.098323 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:44.099536 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:44.101587 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:44.102310 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:44.102437 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:44.106835 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:39:44.106852 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:39:44.106871 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:39:44.106918 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:39:44.110828 (+ 3910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:44.111611 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:44.111620 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:44.111631 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:44.111806 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:44.112183 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:44.112189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:44.112194 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:44.112661 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:39:44.112970 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:44.113305 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 19:39:44.113556 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:39:44.114740 26354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:39:44.207438 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 26219
I20260807 19:39:44.228247 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 26366
Aug 07 19:39:44 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:39:44.246416 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:44.229229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33568 (local address 127.14.16.66:44343)
0807 19:39:44.229413 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:44.229418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:44.229437 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:39:44.229582 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:44.229586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:44.229662 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:44.229750 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:39:44.231521 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:44.232291 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:44.234245 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:44.234630 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:44.234720 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:44.239990 (+ 5270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:39:44.240008 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:39:44.240024 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:39:44.240062 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:39:44.243026 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:44.244106 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:44.244115 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:44.244117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:44.244268 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:44.244684 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:44.244688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:44.244690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:44.245074 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:39:44.245399 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:44.245719 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 19:39:44.245964 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:39:44.247213 26502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:39:44.339402 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 26366
I20260807 19:39:44.362461 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 26513
Aug 07 19:39:44 dist-test-slave-07c2 krb5kdc[25647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:39:44.381942 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:39:44.363502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.14.16.67:39455)
0807 19:39:44.363678 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 19:39:44.363683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:39:44.363700 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:39:44.363820 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:39:44.363824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:39:44.363895 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:39:44.363983 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:39:44.365892 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:44.366723 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:44.368854 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:39:44.369318 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:39:44.369424 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:39:44.375190 (+ 5766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:39:44.375204 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:39:44.375220 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:39:44.375274 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:39:44.378763 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:44.379691 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:44.379699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:44.379701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:44.379866 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:39:44.380321 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:39:44.380329 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:39:44.380335 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:39:44.380731 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:39:44.381025 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 19:39:44.381421 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 19:39:44.381596 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:39:44.382789 26641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:39:44.509640 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 26513
I20260807 19:39:44.531477 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 26103
I20260807 19:39:44.532727 26167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:39:44.649161 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 26103
I20260807 19:39:44.675877 14401 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 19:39:45.020848 14401 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 19:39:45.021080 14401 mini_ranger.cc:66] Stopping Ranger...
I20260807 19:39:45.366010 14401 mini_ranger.cc:68] Stopped Ranger
2026-08-07 19:39:45.366 UTC [25677] LOG: received smart shutdown request
2026-08-07 19:39:45.369 UTC [25677] LOG: background worker "logical replication launcher" (PID 25685) exited with exit code 1
2026-08-07 19:39:45.370 UTC [25680] LOG: shutting down
2026-08-07 19:39:45.529 UTC [25677] LOG: database system is shut down
2026-08-07T19:39:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50370 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:39:45 dist-test-slave-07c2 krb5kdc[26703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:39:45 dist-test-slave-07c2 krb5kdc[26703](info): set up 2 sockets
Aug 07 19:39:45 dist-test-slave-07c2 krb5kdc[26703](info): commencing operation
krb5kdc: starting...
W20260807 19:39:46.814472 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.002s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:39:46 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:39:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:39:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/rangeradmin_127.14.16.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/rangeradmin_127.14.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/rangerlookup_127.14.16.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/rangerlookup_127.14.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/HTTP_127.14.16.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/HTTP_127.14.16.84@KRBTEST.COM.keytab.
I20260807 19:39:47.227888 14401 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskBvS7k8/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/postgres -l logfile start
2026-08-07 19:39:49.700 UTC [26735] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 19:39:49.700 UTC [26735] LOG: listening on IPv4 address "127.14.16.84", port 57559
2026-08-07 19:39:49.704 UTC [26735] LOG: listening on Unix socket "/tmp/.s.PGSQL.57559"
2026-08-07 19:39:49.728 UTC [26738] LOG: database system was shut down at 2026-08-07 19:39:48 UTC
2026-08-07 19:39:49.731 UTC [26735] LOG: database system is ready to accept connections
I20260807 19:39:50.755009 14401 mini_postgres.cc:96] Postgres bound to 57559
2026-08-07 19:39:50.783 UTC [26748] FATAL: database "slave" does not exist
127.14.16.84:57559 - accepting connections
I20260807 19:39:50.784197 14401 mini_ranger.cc:161] Starting Ranger...
I20260807 19:39:50.814517 14401 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 19:39:51.258721 14401 mini_ranger.cc:87] Created ranger Postgres database
I20260807 19:39:51.258860 14401 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-admin
2026-08-07 19:39:51,579 [I] DB FLAVOR :POSTGRES
2026-08-07 19:39:51,580 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:39:51,580 [I] Checking connection..
2026-08-07 19:39:51,580 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 19:39:51,850 [I] Checking connection passed.
2026-08-07 19:39:51,851 [I] --------- Verifying version history table ---------
2026-08-07 19:39:51,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:39:52,120 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:39:52,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:39:52,370 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:39:52,371 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 19:39:52,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 19:39:52.639 UTC [26813] WARNING: there is no transaction in progress
2026-08-07 19:39:52,646 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 19:39:52,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:39:52,915 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 19:39:52,916 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:39:52,916 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 19:39:53,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 19:39:53,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-07c2.c.gcp-upstream.internal','N') ;"
2026-08-07 19:39:53,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:39:53,927 [I] Table x_portal_user does not exist in database ranger
2026-08-07 19:39:53,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:39:54,180 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 19:39:54,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:39:54,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:39:54,685 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 19:39:54,685 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 19:39:55.652 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.669 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.685 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.700 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.715 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.732 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.796 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.802 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.816 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.826 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.836 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.849 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.858 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.864 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.870 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:55.876 UTC [26927] WARNING: there is no transaction in progress
2026-08-07 19:39:56,492 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 19:39:56,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 19:39:56,755 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 19:39:56,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:39:57,020 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 19:39:57,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:39:57,283 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 19:39:57,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:39:57,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:39:57,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-07c2.c.gcp-upstream.internal';"
2026-08-07 19:39:58,070 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 19:39:58,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 19:39:58,339 [I] DB_PATCHES have already been applied
I20260807 19:39:58.346321 14401 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-admin:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskBvS7k8/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 19:39:58.346424 14401 mini_ranger.cc:193] Using host: 127.14.16.84
I20260807 19:39:58.367511 14401 mini_ranger.cc:227] Ranger admin URL: http://127.14.16.84:60799
Aug 07, 2026 7:39:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:39:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:39:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 7:39:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 7:39:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 7:39:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 7:39:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 19:39:59.520334 14401 mini_ranger.cc:160] Time spent starting Ranger: real 8.736s user 0.005s sys 0.055s
Aug 07, 2026 7:40:20 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 7:40:20 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 7:40:20 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 7:40:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 7:40:21 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 19:40:22.582231 14401 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/rangerkms_127.14.16.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/rangerkms_127.14.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.16.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.16.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.16.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/HTTP_127.14.16.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.16.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/HTTP_127.14.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 19:40:22 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131622, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 19:40:22.798589 14401 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 19:40:22.826781 14401 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 19:40:23.600085 14401 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 19:40:23.600188 14401 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-kms
I20260807 19:40:23.600219 14401 mini_ranger_kms.cc:227] Using postgres at 127.14.16.84:57559
2026-08-07 19:40:23,986 [I] DB FLAVOR :POSTGRES
2026-08-07 19:40:23,986 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:40:23,987 [I] Checking connection
2026-08-07 19:40:23,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 19:40:24,229 [I] connection success
2026-08-07 19:40:24,230 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 19:40:24,230 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 19:40:24,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 19:40:24,463 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 19:40:24,464 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:40:24,464 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 19:40:24,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.16.84:57559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 19:40:24,722 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 19:40:25.222872 14401 mini_ranger_kms.cc:310] Created kms service
I20260807 19:40:25.470206 14401 mini_ranger_kms.cc:326] Created kudu user
I20260807 19:40:25.522782 14401 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 19:40:25 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131622, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:40:25.964632 14401 mini_ranger_kms.cc:384] Added ranger policy
I20260807 19:40:25.965013 14401 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-kms:/tmp/dist-test-taskBvS7k8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskBvS7k8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 19:40:25.965077 14401 mini_ranger_kms.cc:240] Using host: 127.14.16.84
I20260807 19:40:25.990298 14401 mini_ranger_kms.cc:276] Ranger KMS PID: 27116
I20260807 19:40:25.990434 14401 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.16.84:48351
Aug 07, 2026 7:40:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:40:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:40:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 7:40:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 7:40:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 7:40:27 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 19:40:28 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131628, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.16.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:28.354420 14401 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.556s user 0.094s sys 0.326s
I20260807 19:40:28.354692 14401 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 19:40:28.354882 14401 mini_ranger_kms.cc:401] 127.14.16.84:48351/kms/v1/keys
Aug 07 19:40:28 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131628, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.16.84@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
Aug 07 19:40:29 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131628, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.16.84@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
Aug 07 19:40:31 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:40:31.394913 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:43489
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:43489
--ranger_config_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.16.84:48351/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:40:31.746410 27167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:31.746766 27167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:31.746923 27167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:31.755553 27167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:40:31.755766 27167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:31.755857 27167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:31.755949 27167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:40:31.756026 27167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:40:31.766499 27167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.16.84:48351/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:43489
--ranger_config_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:43489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:31.767992 27167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:31.769344 27167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:31.781777 27172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:31.781693 27176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:31.781736 27173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:31.782771 27167 server_base.cc:1034] running on GCE node
I20260807 19:40:31.783704 27167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:31.785395 27167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:31.786644 27167 hybrid_clock.cc:648] HybridClock initialized: now 1786131631786598 us; error 34 us; skew 500 ppm
I20260807 19:40:31.787027 27167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:31 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131631, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:31.790933 27167 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:40:31.792642 27167 webserver.cc:460] Webserver started at http://127.14.16.126:46225/ using document root <none> and password file <none>
I20260807 19:40:31.793326 27167 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:31.793452 27167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:31.793794 27167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:40:31 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131631, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:40:31.844053 27167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "ad20dfd3473b436893258cfd36f30695"
format_stamp: "Formatted at 2026-08-07 19:40:31 on dist-test-slave-07c2"
server_key: "0712bf7b2d719367295d7142126c6b01"
server_key_iv: "9ab9e2a23e5947f1fb6eac2e16a02841"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:31.844995 27167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "ad20dfd3473b436893258cfd36f30695"
format_stamp: "Formatted at 2026-08-07 19:40:31 on dist-test-slave-07c2"
server_key: "0712bf7b2d719367295d7142126c6b01"
server_key_iv: "9ab9e2a23e5947f1fb6eac2e16a02841"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:31.850544 27167 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:40:31.866794 27183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:31.868832 27167 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:40:31.869053 27167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "ad20dfd3473b436893258cfd36f30695"
format_stamp: "Formatted at 2026-08-07 19:40:31 on dist-test-slave-07c2"
server_key: "0712bf7b2d719367295d7142126c6b01"
server_key_iv: "9ab9e2a23e5947f1fb6eac2e16a02841"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:31.869376 27167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:31.895459 27167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:31.900660 27167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:31.901031 27167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:31.921793 27167 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:43489
I20260807 19:40:31.921792 27236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:43489 every 8 connection(s)
I20260807 19:40:31.923667 27167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:40:31.927665 27237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:31.928596 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 27167
I20260807 19:40:31.928833 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:40:31.929773 14401 external_mini_cluster.cc:1428] Setting key 2d389551075bb94d03775b683846412b
I20260807 19:40:31.943961 27237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695: Bootstrap starting.
I20260807 19:40:31.948045 27237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695: Neither blocks nor log segments found. Creating new log.
Aug 07 19:40:31 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:31.949405 27237 log.cc:824] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:31.952724 27237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695: No bootstrap required, opened a new log
I20260807 19:40:31.958992 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:31.931996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48658 (local address 127.14.16.126:43489)
0807 19:40:31.932900 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:31.932916 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:31.932969 (+ 53us) server_negotiation.cc:407] Connection header received
0807 19:40:31.935785 (+ 2816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:31.935814 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:31.936349 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:31.936794 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:31.938590 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:31.940706 (+ 2116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:31.942903 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:31.944001 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:31.944206 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:31.949492 (+ 5286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:31.949503 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:31.949523 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:31.949572 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:31.953813 (+ 4241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:31.954738 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:31.954751 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:31.954762 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:31.954962 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:31.955436 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:31.955441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:31.955448 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:31.956152 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:31.956493 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:31.956951 (+ 458us) server_negotiation.cc:299] Negotiation successful
0807 19:40:31.957565 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":408,"threads_started":1}
I20260807 19:40:31.961997 27237 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad20dfd3473b436893258cfd36f30695" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43489 } }
I20260807 19:40:31.962474 27237 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:31.962586 27237 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad20dfd3473b436893258cfd36f30695, State: Initialized, Role: FOLLOWER
I20260807 19:40:31.963213 27237 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad20dfd3473b436893258cfd36f30695" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43489 } }
I20260807 19:40:31.963513 27237 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:40:31.963663 27237 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:40:31.963836 27237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:31.966367 27237 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad20dfd3473b436893258cfd36f30695" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43489 } }
I20260807 19:40:31.967017 27237 leader_election.cc:302] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad20dfd3473b436893258cfd36f30695; no voters:
I20260807 19:40:31.967658 27237 leader_election.cc:288] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:40:31.967808 27242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:40:31.968555 27242 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 1 LEADER]: Becoming Leader. State: Replica: ad20dfd3473b436893258cfd36f30695, State: Running, Role: LEADER
I20260807 19:40:31.969130 27242 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad20dfd3473b436893258cfd36f30695" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43489 } }
I20260807 19:40:31.969836 27237 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:40:31.972185 27244 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad20dfd3473b436893258cfd36f30695. Latest consensus state: current_term: 1 leader_uuid: "ad20dfd3473b436893258cfd36f30695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad20dfd3473b436893258cfd36f30695" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43489 } } }
I20260807 19:40:31.972568 27244 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:31.972309 27243 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad20dfd3473b436893258cfd36f30695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad20dfd3473b436893258cfd36f30695" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43489 } } }
I20260807 19:40:31.972812 27243 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:31.975791 27247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:40:31.980990 27247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:40:31.981580 27237 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 19:40:32.000730 27247 catalog_manager.cc:1347] Generated new cluster ID: 8bdb8b5bf9924a528d70da2c0b2b1044
I20260807 19:40:32.000876 27247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:40:32.019788 27247 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:40:32.021731 27247 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:40:32.031349 27247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695: Generated new TSK 0
I20260807 19:40:32.032466 27247 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:40:32.775504 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.16.84:48351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43489
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:40:33 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 19:40:33.254211 27284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:33.254619 27284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:33.254841 27284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:33.266726 27284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:33.266887 27284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:33.267134 27284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:40:33.284050 27284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.16.84:48351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:33.285856 27284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:33.287670 27284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:33.304415 27294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:33.306878 27295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:33.310303 27284 server_base.cc:1034] running on GCE node
W20260807 19:40:33.311025 27297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:33.311985 27284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:33.321135 27284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:33.324599 27284 hybrid_clock.cc:648] HybridClock initialized: now 1786131633324478 us; error 116 us; skew 500 ppm
I20260807 19:40:33.325086 27284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:33 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:33.329504 27284 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:40:33.331548 27284 webserver.cc:460] Webserver started at http://127.14.16.65:41253/ using document root <none> and password file <none>
I20260807 19:40:33.332432 27284 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:33.332580 27284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:33.332947 27284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:40:33 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:40:33.354390 27284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "0775d1be5ac64423a5fddc0454254038"
format_stamp: "Formatted at 2026-08-07 19:40:33 on dist-test-slave-07c2"
server_key: "489a50b855c54e9ec2b902185b510190"
server_key_iv: "b9fb580359315c6da93325e9ee2845a2"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:33.355427 27284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "0775d1be5ac64423a5fddc0454254038"
format_stamp: "Formatted at 2026-08-07 19:40:33 on dist-test-slave-07c2"
server_key: "489a50b855c54e9ec2b902185b510190"
server_key_iv: "b9fb580359315c6da93325e9ee2845a2"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:33.365063 27284 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260807 19:40:33.393035 27304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:33.395092 27284 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.008s
I20260807 19:40:33.395378 27284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "0775d1be5ac64423a5fddc0454254038"
format_stamp: "Formatted at 2026-08-07 19:40:33 on dist-test-slave-07c2"
server_key: "489a50b855c54e9ec2b902185b510190"
server_key_iv: "b9fb580359315c6da93325e9ee2845a2"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:33.395677 27284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:33.454314 27284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:33.459208 27284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:33.459681 27284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:33.461315 27284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:33.462855 27284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:33.462960 27284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:33.463160 27284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:33.463260 27284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:33.528824 27284 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:33497
I20260807 19:40:33.528878 27421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:33497 every 8 connection(s)
I20260807 19:40:33.530510 27284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:40:33.530948 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 27284
I20260807 19:40:33.531172 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:40:33.532071 14401 external_mini_cluster.cc:1428] Setting key 62b07a927fef64b4e8932832717b2bba
Aug 07 19:40:33 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
Aug 07 19:40:33 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:40:33.599153 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:33.540522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:38853 (local address 127.14.16.126:43489)
0807 19:40:33.570455 (+ 29933us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:33.570461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:33.573964 (+ 3503us) server_negotiation.cc:407] Connection header received
0807 19:40:33.577169 (+ 3205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:33.577175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:33.577290 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:33.577377 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:33.581912 (+ 4535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:33.583069 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:33.585754 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:33.586404 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:33.586516 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:33.591396 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:33.591404 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:33.591410 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:33.591453 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:33.595088 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:33.596105 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:33.596110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:33.596114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:33.596295 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:33.596865 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:33.596870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:33.596873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:33.597259 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:33.597398 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:33.598510 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0807 19:40:33.598761 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29764,"thread_start_us":29956,"threads_started":1}
I20260807 19:40:33.602046 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:33.539929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:43489 (local address 127.14.16.65:38853)
0807 19:40:33.572370 (+ 32441us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:33.572433 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:33.576806 (+ 4373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:33.577982 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:33.577994 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:33.578698 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:33.581665 (+ 2967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:33.581700 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:33.583341 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:33.583346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:33.585554 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:33.585569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:33.586905 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:33.586913 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:33.587238 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:33.588086 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:33.588121 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:33.591051 (+ 2930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:33.595242 (+ 4191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:33.595252 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:33.595272 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:33.595957 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:33.596444 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:33.596450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:33.596456 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:33.596717 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:33.597504 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:33.597511 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:33.597985 (+ 474us) client_negotiation.cc:769] Sending connection context
0807 19:40:33.599849 (+ 1864us) client_negotiation.cc:240] Negotiation successful
0807 19:40:33.600443 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32042,"thread_start_us":34728,"threads_started":1}
I20260807 19:40:33.604324 27422 heartbeater.cc:347] Connected to a master server at 127.14.16.126:43489
I20260807 19:40:33.604672 27422 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:33.605577 27422 heartbeater.cc:511] Master 127.14.16.126:43489 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
I20260807 19:40:33.608425 27202 ts_manager.cc:194] Registered new tserver with Master: 0775d1be5ac64423a5fddc0454254038 (127.14.16.65:33497)
I20260807 19:40:33.611042 27202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:38853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
W20260807 19:40:33.653658 14401 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.14.16.66: real 0.114s user 0.007s sys 0.017s
I20260807 19:40:33.654572 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.16.84:48351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43489
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:40:33 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131633, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
W20260807 19:40:34.131925 27429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:34.132468 27429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:34.132695 27429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:34.144404 27429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:34.144688 27429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:34.144917 27429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:40:34.160835 27429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.16.84:48351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:34.162812 27429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:34.164644 27429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:34.187359 27436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:34.189070 27437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:34.194285 27429 server_base.cc:1034] running on GCE node
W20260807 19:40:34.194697 27439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:34.195652 27429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:34.199489 27429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:34.201009 27429 hybrid_clock.cc:648] HybridClock initialized: now 1786131634200897 us; error 104 us; skew 500 ppm
I20260807 19:40:34.201571 27429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:34 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131634, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:34.207039 27429 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:40:34.215088 27429 webserver.cc:460] Webserver started at http://127.14.16.66:33025/ using document root <none> and password file <none>
I20260807 19:40:34.215950 27429 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:34.216110 27429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:34.216468 27429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:40:34 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131634, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:40:34.240613 27429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c"
format_stamp: "Formatted at 2026-08-07 19:40:34 on dist-test-slave-07c2"
server_key: "81361daf17feca0f32be70d7bd54d6c3"
server_key_iv: "0ae0feb681f58583bd37f6ae949e1f6c"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:34.241833 27429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c"
format_stamp: "Formatted at 2026-08-07 19:40:34 on dist-test-slave-07c2"
server_key: "81361daf17feca0f32be70d7bd54d6c3"
server_key_iv: "0ae0feb681f58583bd37f6ae949e1f6c"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:34.259600 27429 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.004s sys 0.012s
I20260807 19:40:34.279410 27450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:34.281574 27429 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:40:34.281903 27429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c"
format_stamp: "Formatted at 2026-08-07 19:40:34 on dist-test-slave-07c2"
server_key: "81361daf17feca0f32be70d7bd54d6c3"
server_key_iv: "0ae0feb681f58583bd37f6ae949e1f6c"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:34.282306 27429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:34.310667 27429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:34.316128 27429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:34.316478 27429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:34.317754 27429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:34.319697 27429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:34.319809 27429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:34.319996 27429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:34.320067 27429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:34.383852 27429 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:42287
I20260807 19:40:34.383895 27569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:42287 every 8 connection(s)
I20260807 19:40:34.385603 27429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:40:34.390245 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 27429
I20260807 19:40:34.390410 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:40:34.390940 14401 external_mini_cluster.cc:1428] Setting key ab1c37853dd4e02518945afd977efce9
Aug 07 19:40:34 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131634, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:34.415930 27573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:34.389606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:47933 (local address 127.14.16.126:43489)
0807 19:40:34.392056 (+ 2450us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:34.392062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:34.395473 (+ 3411us) server_negotiation.cc:407] Connection header received
0807 19:40:34.395539 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:34.395544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:34.395633 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:34.395744 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:34.399390 (+ 3646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:34.400476 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:34.402918 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:34.403595 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:34.403698 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:34.408494 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:34.408500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:34.408506 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:34.408570 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:34.412229 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:34.413164 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:34.413170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:34.413173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:34.413315 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:34.413813 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:34.413818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:34.413821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:34.414194 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:34.414309 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:34.415313 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0807 19:40:34.415553 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2320,"thread_start_us":3540,"threads_started":1}
I20260807 19:40:34.418182 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:34.389053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:43489 (local address 127.14.16.66:47933)
0807 19:40:34.392238 (+ 3185us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:34.392292 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:34.394766 (+ 2474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:34.397634 (+ 2868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:34.397646 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:34.398353 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:34.399140 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:34.399165 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:34.400698 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:34.400704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:34.402745 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:34.402761 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:34.404053 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:34.404059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:34.404341 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:34.405134 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:34.405165 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:34.408221 (+ 3056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:34.412402 (+ 4181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:34.412411 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:34.412429 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:34.413010 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:34.413449 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:34.413454 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:34.413457 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:34.413689 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:34.414414 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:34.414422 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:34.414840 (+ 418us) client_negotiation.cc:769] Sending connection context
0807 19:40:34.416242 (+ 1402us) client_negotiation.cc:240] Negotiation successful
0807 19:40:34.416801 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1994,"thread_start_us":8477,"threads_started":1}
I20260807 19:40:34.420010 27570 heartbeater.cc:347] Connected to a master server at 127.14.16.126:43489
I20260807 19:40:34.420332 27570 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:34.421177 27570 heartbeater.cc:511] Master 127.14.16.126:43489 requested a full tablet report, sending...
I20260807 19:40:34.422855 27202 ts_manager.cc:194] Registered new tserver with Master: 7abd0a2cc3744c11b3738f2e73cb0f4c (127.14.16.66:42287)
I20260807 19:40:34.423977 27202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:47933
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:40:34.462375 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.16.84:48351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43489
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:40:34.614766 27422 heartbeater.cc:503] Master 127.14.16.126:43489 was elected leader, sending a full tablet report...
W20260807 19:40:34.813498 27576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:34.813792 27576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:34.813889 27576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:34.821372 27576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:34.821492 27576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:34.821619 27576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:40:34.831714 27576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.16.84:48351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:34.832984 27576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:34.834270 27576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:34.847716 27581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:34.847668 27584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:34.847762 27582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:34.848656 27576 server_base.cc:1034] running on GCE node
I20260807 19:40:34.849241 27576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:34.850174 27576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:34.851400 27576 hybrid_clock.cc:648] HybridClock initialized: now 1786131634851355 us; error 41 us; skew 500 ppm
I20260807 19:40:34.851758 27576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:34 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131634, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:34.854993 27576 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:40:34.856627 27576 webserver.cc:460] Webserver started at http://127.14.16.67:46867/ using document root <none> and password file <none>
I20260807 19:40:34.857342 27576 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:34.857447 27576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:34.857817 27576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:40:34 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131634, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:40:34.874945 27576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "e7013f40bccb49a1b90088e5e4adb7c8"
format_stamp: "Formatted at 2026-08-07 19:40:34 on dist-test-slave-07c2"
server_key: "68505f2c1020269680158106c6f6bd8a"
server_key_iv: "df36a83f9de8df91490ad978240deb2f"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:34.876083 27576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "e7013f40bccb49a1b90088e5e4adb7c8"
format_stamp: "Formatted at 2026-08-07 19:40:34 on dist-test-slave-07c2"
server_key: "68505f2c1020269680158106c6f6bd8a"
server_key_iv: "df36a83f9de8df91490ad978240deb2f"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:34.882972 27576 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 19:40:34.899554 27591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:34.901260 27576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:40:34.901538 27576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "e7013f40bccb49a1b90088e5e4adb7c8"
format_stamp: "Formatted at 2026-08-07 19:40:34 on dist-test-slave-07c2"
server_key: "68505f2c1020269680158106c6f6bd8a"
server_key_iv: "df36a83f9de8df91490ad978240deb2f"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:34.901803 27576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:34.916443 27576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:34.920672 27576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:34.920975 27576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:34.922114 27576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:34.923691 27576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:34.923800 27576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:34.923957 27576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:34.924041 27576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:34.958770 27576 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:41257
I20260807 19:40:34.958806 27707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:41257 every 8 connection(s)
I20260807 19:40:34.960151 27576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:40:34.962656 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 27576
I20260807 19:40:34.962869 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:40:34.963526 14401 external_mini_cluster.cc:1428] Setting key 427a75063a0a0cbcaa3fab2cecdc97a0
Aug 07 19:40:34 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131634, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:34.980089 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:34.962725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:59561 (local address 127.14.16.126:43489)
0807 19:40:34.963305 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:34.963312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:34.964331 (+ 1019us) server_negotiation.cc:407] Connection header received
0807 19:40:34.966162 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:34.966168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:34.966293 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:34.966412 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:34.967908 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:34.968647 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:34.970217 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:34.970624 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:34.970720 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:34.973964 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:34.973970 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:34.973974 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:34.974006 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:34.976843 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:34.977718 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:34.977742 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:34.977748 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:34.977925 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:34.978461 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:34.978466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:34.978469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:34.978837 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:34.978977 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:34.979673 (+ 696us) server_negotiation.cc:299] Negotiation successful
0807 19:40:34.979839 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":259,"threads_started":1}
I20260807 19:40:34.981014 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:34.963267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:43489 (local address 127.14.16.67:59561)
0807 19:40:34.964151 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:34.964204 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:34.965894 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:34.966660 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:34.966674 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:34.967118 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:34.967759 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:34.967778 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:34.968788 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:34.968792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:34.970063 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:34.970075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:34.970826 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:34.970831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:34.971012 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:34.971716 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:34.971747 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:34.973764 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:34.976975 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:34.976983 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:34.976997 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:34.977604 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:34.978040 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:34.978044 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:34.978055 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:34.978321 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:34.978968 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:34.978976 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:34.979279 (+ 303us) client_negotiation.cc:769] Sending connection context
0807 19:40:34.979570 (+ 291us) client_negotiation.cc:240] Negotiation successful
0807 19:40:34.979955 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":390,"threads_started":1}
I20260807 19:40:34.982456 27708 heartbeater.cc:347] Connected to a master server at 127.14.16.126:43489
I20260807 19:40:34.982769 27708 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:34.983520 27708 heartbeater.cc:511] Master 127.14.16.126:43489 requested a full tablet report, sending...
I20260807 19:40:34.984934 27202 ts_manager.cc:194] Registered new tserver with Master: e7013f40bccb49a1b90088e5e4adb7c8 (127.14.16.67:41257)
I20260807 19:40:34.985765 27202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:59561
I20260807 19:40:34.991215 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:40:34.993021 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 27284
Aug 07 19:40:34 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:40:35.009544 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:34.993981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.14.16.65:33497)
0807 19:40:34.994602 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:34.994608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:34.994632 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:40:34.994686 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:34.994692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:34.994874 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:34.995002 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:34.996525 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:34.997559 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:34.998949 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:34.999528 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:34.999627 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:35.003805 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:35.003816 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:35.003829 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:35.003867 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:35.006921 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:35.007617 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:35.007623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:35.007625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:35.007788 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:35.008101 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:35.008104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:35.008106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:35.008423 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:35.008672 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:35.009056 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 19:40:35.009264 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":320,"threads_started":1}
I20260807 19:40:35.010814 27416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:35.099282 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 27284
I20260807 19:40:35.127254 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 27429
Aug 07 19:40:35 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:40:35.148393 27715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:35.128379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37066 (local address 127.14.16.66:42287)
0807 19:40:35.129278 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:35.129285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:35.129311 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:40:35.129372 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:35.129379 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:35.129618 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:35.129750 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:35.131810 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:35.132795 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:35.135343 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:35.135906 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:35.136005 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:35.142214 (+ 6209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:35.142229 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:35.142244 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:35.142289 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:35.145338 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:35.146070 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:35.146078 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:35.146080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:35.146231 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:35.146618 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:35.146623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:35.146627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:35.146994 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:35.147446 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:35.147674 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 19:40:35.147995 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":615,"threads_started":1}
I20260807 19:40:35.149623 27564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:35.306273 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 27429
I20260807 19:40:35.324026 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 27576
Aug 07 19:40:35 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:40:35.341676 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:35.325020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40722 (local address 127.14.16.67:41257)
0807 19:40:35.325861 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:35.325868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:35.325896 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:40:35.325955 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:35.325962 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:35.326164 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:35.326358 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:35.327733 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:35.328770 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:35.330233 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:35.330803 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:35.330945 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:35.335819 (+ 4874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:35.335830 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:35.335844 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:35.335883 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:35.338774 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:35.339541 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:35.339557 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:35.339559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:35.339690 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:35.340091 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:35.340095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:35.340097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:35.340417 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:35.340749 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:35.341059 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 19:40:35.341332 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":505,"threads_started":1}
I20260807 19:40:35.343071 27703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:35.427822 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 27576
I20260807 19:40:35.446012 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 27167
I20260807 19:40:35.447268 27232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:35.556566 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 27167
I20260807 19:40:35.587601 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:43489
--webserver_interface=127.14.16.126
--webserver_port=46225
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:43489
--ranger_config_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.16.84:48351/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.16.84:48351/kms with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:40:35.922047 27720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:35.922418 27720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:35.922610 27720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:35.930131 27720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:40:35.930253 27720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:35.930377 27720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:35.930474 27720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:40:35.930568 27720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:40:35.940815 27720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.16.84:48351/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:43489
--ranger_config_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:43489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=46225
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:35.942114 27720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:35.943495 27720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:35.955608 27725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:35.955531 27728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:35.955561 27726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:35.956535 27720 server_base.cc:1034] running on GCE node
I20260807 19:40:35.957296 27720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:35.958595 27720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:35.959789 27720 hybrid_clock.cc:648] HybridClock initialized: now 1786131635959727 us; error 51 us; skew 500 ppm
I20260807 19:40:35.960098 27720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:35 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131635, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:35.965845 27720 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:40:35.967198 27720 webserver.cc:460] Webserver started at http://127.14.16.126:46225/ using document root <none> and password file <none>
I20260807 19:40:35.967875 27720 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:35.967984 27720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:40:36 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131635, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:40:36.044294 27720 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:40:36.047374 27736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:36.048805 27720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:40:36.048961 27720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "ad20dfd3473b436893258cfd36f30695"
format_stamp: "Formatted at 2026-08-07 19:40:31 on dist-test-slave-07c2"
server_key: "0712bf7b2d719367295d7142126c6b01"
server_key_iv: "9ab9e2a23e5947f1fb6eac2e16a02841"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:36.049893 27720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:36.069150 27720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:36.073429 27720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:36.073680 27720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:36.093158 27720 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:43489
I20260807 19:40:36.093183 27792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:43489 every 8 connection(s)
I20260807 19:40:36.094707 27720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:40:36.097003 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 27720
I20260807 19:40:36.098086 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.16.84:48351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:33497
--local_ip_for_outbound_sockets=127.14.16.65
--tserver_master_addrs=127.14.16.126:43489
--webserver_port=41253
--webserver_interface=127.14.16.65
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.16.84:48351/kms with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:40:36.104549 27793 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:40:36.109879 27793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695: Bootstrap starting.
I20260807 19:40:36.129555 27793 log.cc:824] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:36.138836 27793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:40:36.139528 27793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695: Bootstrap complete.
I20260807 19:40:36.148792 27793 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad20dfd3473b436893258cfd36f30695" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43489 } }
I20260807 19:40:36.150095 27793 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad20dfd3473b436893258cfd36f30695, State: Initialized, Role: FOLLOWER
I20260807 19:40:36.150723 27793 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad20dfd3473b436893258cfd36f30695" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43489 } }
I20260807 19:40:36.150967 27793 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:40:36.151093 27793 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:40:36.151315 27793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:40:36.155186 27793 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad20dfd3473b436893258cfd36f30695" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43489 } }
I20260807 19:40:36.155778 27793 leader_election.cc:302] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad20dfd3473b436893258cfd36f30695; no voters:
I20260807 19:40:36.156455 27793 leader_election.cc:288] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:40:36.156601 27797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:40:36.159108 27793 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:40:36.163717 27797 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [term 2 LEADER]: Becoming Leader. State: Replica: ad20dfd3473b436893258cfd36f30695, State: Running, Role: LEADER
I20260807 19:40:36.164392 27797 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad20dfd3473b436893258cfd36f30695" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43489 } }
I20260807 19:40:36.167557 27799 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad20dfd3473b436893258cfd36f30695. Latest consensus state: current_term: 2 leader_uuid: "ad20dfd3473b436893258cfd36f30695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad20dfd3473b436893258cfd36f30695" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43489 } } }
I20260807 19:40:36.168018 27799 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:36.168503 27798 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ad20dfd3473b436893258cfd36f30695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad20dfd3473b436893258cfd36f30695" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43489 } } }
I20260807 19:40:36.168855 27798 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:36.176189 27812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:40:36.181648 27812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:40:36.183048 27812 catalog_manager.cc:1259] Loaded cluster ID: 8bdb8b5bf9924a528d70da2c0b2b1044
I20260807 19:40:36.183161 27812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:40:36.187220 27812 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:40:36.188905 27812 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695: Loaded TSK: 0
I20260807 19:40:36.190449 27812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:40:36.192907 27793 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260807 19:40:36.520744 27795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:36.521119 27795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:36.521310 27795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:36.532931 27795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:36.533089 27795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:36.533283 27795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:40:36.548386 27795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.16.84:48351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:33497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=41253
--tserver_master_addrs=127.14.16.126:43489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:36.550356 27795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:36.552253 27795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:36.592018 27832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:36.592564 27831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:36.596156 27834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:36.598767 27795 server_base.cc:1034] running on GCE node
I20260807 19:40:36.599694 27795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:36.607318 27795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:36.608711 27795 hybrid_clock.cc:648] HybridClock initialized: now 1786131636608626 us; error 73 us; skew 500 ppm
I20260807 19:40:36.609167 27795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:36 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131636, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:36.613149 27795 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:40:36.617252 27795 webserver.cc:460] Webserver started at http://127.14.16.65:41253/ using document root <none> and password file <none>
I20260807 19:40:36.618181 27795 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:36.618378 27795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:40:36 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131636, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:40:36.653209 27795 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:40:36.657030 27842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:36.658509 27795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:40:36.658707 27795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "0775d1be5ac64423a5fddc0454254038"
format_stamp: "Formatted at 2026-08-07 19:40:33 on dist-test-slave-07c2"
server_key: "489a50b855c54e9ec2b902185b510190"
server_key_iv: "b9fb580359315c6da93325e9ee2845a2"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:36.660419 27795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:36.678150 27795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:36.683426 27795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:36.683847 27795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:36.691347 27795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:36.712294 27795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:36.712446 27795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:36.712581 27795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:36.712671 27795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:36.853619 27795 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:33497
I20260807 19:40:36.854002 27962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:33497 every 8 connection(s)
I20260807 19:40:36.855676 27795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:40:36.864725 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 27795
I20260807 19:40:36.865595 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.16.84:48351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:42287
--local_ip_for_outbound_sockets=127.14.16.66
--tserver_master_addrs=127.14.16.126:43489
--webserver_port=33025
--webserver_interface=127.14.16.66
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.16.84:48351/kms with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:40:36 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131636, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:36.888777 27965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:36.858943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:43489 (local address 127.14.16.65:58917)
0807 19:40:36.859871 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:36.859917 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:36.862721 (+ 2804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:36.864186 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:36.864197 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:36.864756 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:36.865377 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:36.865401 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:36.867084 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:36.867089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:36.868329 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:36.868341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:36.869369 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:36.869376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:36.869646 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:36.870336 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:36.870358 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:36.880322 (+ 9964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:36.884474 (+ 4152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:36.884483 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:36.884499 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:36.884925 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:36.885378 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:36.885385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:36.885389 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:36.885601 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:36.886610 (+ 1009us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:36.886615 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:36.886914 (+ 299us) client_negotiation.cc:769] Sending connection context
0807 19:40:36.887308 (+ 394us) client_negotiation.cc:240] Negotiation successful
0807 19:40:36.887766 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":857,"threads_started":1}
I20260807 19:40:36.890805 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:36.860132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:58917 (local address 127.14.16.126:43489)
0807 19:40:36.862338 (+ 2206us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:36.862357 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:36.862415 (+ 58us) server_negotiation.cc:407] Connection header received
0807 19:40:36.863361 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:36.863375 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:36.863701 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:36.864044 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:36.865610 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:36.866914 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:36.868491 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:36.869138 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:36.869335 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:36.880665 (+ 11330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:36.880677 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:36.880696 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:36.880748 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:36.884321 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:36.885097 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:36.885104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:36.885109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:36.885256 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:36.885734 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:36.885738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:36.885740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:36.886274 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:36.886522 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:36.889506 (+ 2984us) server_negotiation.cc:299] Negotiation successful
0807 19:40:36.889854 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1944,"thread_start_us":1710,"threads_started":1}
I20260807 19:40:36.894121 27963 heartbeater.cc:347] Connected to a master server at 127.14.16.126:43489
I20260807 19:40:36.894431 27963 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:36.895164 27963 heartbeater.cc:511] Master 127.14.16.126:43489 requested a full tablet report, sending...
W20260807 19:40:36.898020 27963 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:43489 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:40:36.898257 27963 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:36.898514 27963 heartbeater.cc:511] Master 127.14.16.126:43489 requested a full tablet report, sending...
I20260807 19:40:36.899863 27963 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:36.900127 27963 heartbeater.cc:511] Master 127.14.16.126:43489 requested a full tablet report, sending...
W20260807 19:40:36.901189 27963 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:40:37.281662 27967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:37.282009 27967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:37.282209 27967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:37.292157 27967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:37.292320 27967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:37.292507 27967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:40:37.307634 27967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.16.84:48351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:42287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=33025
--tserver_master_addrs=127.14.16.126:43489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:37.309056 27967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:37.310822 27967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:37.328171 27982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:37.328692 27983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:37.331053 27985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:37.331954 27967 server_base.cc:1034] running on GCE node
I20260807 19:40:37.332674 27967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:37.333788 27967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:37.335041 27967 hybrid_clock.cc:648] HybridClock initialized: now 1786131637334980 us; error 52 us; skew 500 ppm
I20260807 19:40:37.335489 27967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:37 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:37.338944 27967 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:40:37.344707 27967 webserver.cc:460] Webserver started at http://127.14.16.66:33025/ using document root <none> and password file <none>
I20260807 19:40:37.345479 27967 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:37.345649 27967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:40:37 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:40:37.388530 27967 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 19:40:37.394155 27995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:37.408887 27967 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.006s sys 0.001s
I20260807 19:40:37.409124 27967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c"
format_stamp: "Formatted at 2026-08-07 19:40:34 on dist-test-slave-07c2"
server_key: "81361daf17feca0f32be70d7bd54d6c3"
server_key_iv: "0ae0feb681f58583bd37f6ae949e1f6c"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:37.410626 27967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 07 19:40:37 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:37.500597 27967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:37.505754 27967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:37.506151 27967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:37.510324 27967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:37.511991 27967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:37.512135 27967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 19:40:37.512322 27967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:37.512452 27967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:37.565784 27967 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:42287
I20260807 19:40:37.565840 28114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:42287 every 8 connection(s)
I20260807 19:40:37.567376 27967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:40:37.574792 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 27967
I20260807 19:40:37.575742 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.16.84:48351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:41257
--local_ip_for_outbound_sockets=127.14.16.67
--tserver_master_addrs=127.14.16.126:43489
--webserver_port=46867
--webserver_interface=127.14.16.67
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.16.84:48351/kms with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:40:37 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:37.596889 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:37.572298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:33501 (local address 127.14.16.126:43489)
0807 19:40:37.578892 (+ 6594us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:37.578901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:37.578927 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:40:37.578999 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:37.579004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:37.579082 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:37.579213 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:37.581179 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:37.582358 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:37.584882 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:37.585642 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:37.585783 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:37.590334 (+ 4551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:37.590341 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:37.590347 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:37.590396 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:37.593478 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:37.594240 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:37.594246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:37.594249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:37.594466 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:37.594951 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:37.594955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:37.594959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:37.595250 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:37.595453 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:37.596247 (+ 794us) server_negotiation.cc:299] Negotiation successful
0807 19:40:37.596483 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6438,"thread_start_us":6312,"threads_started":1}
I20260807 19:40:37.597983 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:37.571515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:43489 (local address 127.14.16.66:33501)
0807 19:40:37.573985 (+ 2470us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:37.574034 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:37.577503 (+ 3469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:37.579914 (+ 2411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:37.579923 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:37.580398 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:37.580977 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:37.581004 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:37.582556 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:37.582561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:37.584672 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:37.584687 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:37.586135 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:37.586144 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:37.586458 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:37.587274 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:37.587336 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:37.590065 (+ 2729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:37.593637 (+ 3572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:37.593646 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:37.593664 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:37.594100 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:37.594590 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:37.594595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:37.594599 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:37.594823 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:37.595515 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:37.595521 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:37.595862 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 19:40:37.596133 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 19:40:37.596725 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2164,"thread_start_us":1883,"threads_started":1}
I20260807 19:40:37.599804 28115 heartbeater.cc:347] Connected to a master server at 127.14.16.126:43489
I20260807 19:40:37.600140 28115 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:37.601114 28115 heartbeater.cc:511] Master 127.14.16.126:43489 requested a full tablet report, sending...
I20260807 19:40:37.603475 27753 ts_manager.cc:194] Registered new tserver with Master: 7abd0a2cc3744c11b3738f2e73cb0f4c (127.14.16.66:42287)
I20260807 19:40:37.605937 27753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:33501
I20260807 19:40:37.901407 27963 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:37.901889 27963 heartbeater.cc:511] Master 127.14.16.126:43489 requested a full tablet report, sending...
I20260807 19:40:37.903373 27753 ts_manager.cc:194] Registered new tserver with Master: 0775d1be5ac64423a5fddc0454254038 (127.14.16.65:33497)
I20260807 19:40:37.904544 27753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:58917
Aug 07 19:40:37 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
W20260807 19:40:38.040815 28120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:38.041332 28120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:38.041519 28120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:38.052932 28120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:38.053164 28120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:38.053386 28120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:40:38.068028 28120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.16.84:48351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:41257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=46867
--tserver_master_addrs=127.14.16.126:43489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:38.069986 28120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:38.071836 28120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:38.130259 28126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:38.131647 28125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:38.137133 28130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:38.138407 28120 server_base.cc:1034] running on GCE node
I20260807 19:40:38.139444 28120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:38.148036 28120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:38.149529 28120 hybrid_clock.cc:648] HybridClock initialized: now 1786131638149449 us; error 79 us; skew 500 ppm
I20260807 19:40:38.150022 28120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:38 dist-test-slave-07c2 krb5kdc[26703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131638, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:38.154558 28120 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:40:38.175139 28120 webserver.cc:460] Webserver started at http://127.14.16.67:46867/ using document root <none> and password file <none>
I20260807 19:40:38.176332 28120 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:38.176600 28120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:40:38 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131638, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:40:38.212620 28120 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.009s sys 0.001s
I20260807 19:40:38.223042 28137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:38.225072 28120 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.005s
I20260807 19:40:38.225329 28120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "e7013f40bccb49a1b90088e5e4adb7c8"
format_stamp: "Formatted at 2026-08-07 19:40:34 on dist-test-slave-07c2"
server_key: "68505f2c1020269680158106c6f6bd8a"
server_key_iv: "df36a83f9de8df91490ad978240deb2f"
server_key_version: "kuduclusterkey@0"
I20260807 19:40:38.226876 28120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:38.276979 28120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:38.282598 28120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:38.283028 28120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:38.290697 28120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:38.299099 28120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:38.299338 28120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:38.299532 28120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:38.299662 28120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:40:38 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131637, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for HTTP/127.14.16.84@KRBTEST.COM
I20260807 19:40:38.576164 28120 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:41257
I20260807 19:40:38.576241 28256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:41257 every 8 connection(s)
I20260807 19:40:38.578243 28120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:40:38.582659 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 28120
Aug 07 19:40:38 dist-test-slave-07c2 krb5kdc[26703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131638, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:38.611582 28115 heartbeater.cc:503] Master 127.14.16.126:43489 was elected leader, sending a full tablet report...
I20260807 19:40:38.616111 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.597963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53968 (local address 127.14.16.126:43489)
0807 19:40:38.600035 (+ 2072us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:38.600039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:38.600055 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:40:38.600098 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:38.600101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:38.600166 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:38.600291 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:38.602173 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.603445 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.605134 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.605841 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.605982 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.609211 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:38.609216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:38.609219 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:38.609254 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:38.613102 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:38.613923 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:38.613931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:38.613936 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:38.614123 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:38.614508 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:38.614513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:38.614516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:38.614780 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:38.615058 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:38.615501 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 19:40:38.615725 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1970,"thread_start_us":1806,"threads_started":1}
I20260807 19:40:38.616556 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.582245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:51583 (local address 127.14.16.126:43489)
0807 19:40:38.597935 (+ 15690us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:38.597942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:38.597965 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:40:38.598030 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:38.598036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:38.598152 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:38.598271 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:38.601447 (+ 3176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.602453 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.604488 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.605144 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.605280 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.609290 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:38.609295 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:38.609298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:38.609345 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:38.613204 (+ 3859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:38.614030 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:38.614035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:38.614040 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:38.614167 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:38.614641 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:38.614645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:38.614647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:38.614896 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:38.615054 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:38.615826 (+ 772us) server_negotiation.cc:299] Negotiation successful
0807 19:40:38.616029 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":15557,"thread_start_us":15440,"threads_started":1}
I20260807 19:40:38.617622 28265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.582628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:43489 (local address 127.14.16.67:51583)
0807 19:40:38.594401 (+ 11773us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:38.594468 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:38.597074 (+ 2606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:38.599678 (+ 2604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:38.599692 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:38.600396 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:38.601226 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:38.601251 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.602636 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.602646 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:38.604318 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:38.604335 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.605384 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.605391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:38.605706 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.606610 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:38.606656 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:38.609003 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:38.613367 (+ 4364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:38.613376 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:38.613394 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:38.613902 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:38.614283 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:38.614288 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:38.614292 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:38.614506 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:38.615029 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:38.615037 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:38.615463 (+ 426us) client_negotiation.cc:769] Sending connection context
0807 19:40:38.615737 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 19:40:38.616300 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11359,"thread_start_us":15627,"threads_started":1}
I20260807 19:40:38.619168 28259 heartbeater.cc:347] Connected to a master server at 127.14.16.126:43489
I20260807 19:40:38.619505 28259 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:38.620227 28259 heartbeater.cc:511] Master 127.14.16.126:43489 requested a full tablet report, sending...
I20260807 19:40:38.621635 27752 ts_manager.cc:194] Registered new tserver with Master: e7013f40bccb49a1b90088e5e4adb7c8 (127.14.16.67:41257)
I20260807 19:40:38.622530 27752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:51583
I20260807 19:40:38.631201 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:40:38.647560 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.634363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53970 (local address 127.14.16.126:43489)
0807 19:40:38.634581 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:38.634590 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:38.634962 (+ 372us) server_negotiation.cc:407] Connection header received
0807 19:40:38.635192 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:38.635196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:38.635256 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:38.635381 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:38.636957 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.637720 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.639142 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.639628 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.639769 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.642355 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:38.642360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:38.642363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:38.642396 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:38.644898 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:38.645571 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:38.645577 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:38.645581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:38.645721 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:38.646116 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:38.646121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:38.646124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:38.646377 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:38.646622 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:38.646957 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 19:40:38.647199 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:40:38.652493 27752 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:40:38.660686 27752 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:40:38.685381 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.676197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47198 (local address 127.14.16.66:42287)
0807 19:40:38.677999 (+ 1802us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:38.678007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:38.678034 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:40:38.678138 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:38.678145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:38.678375 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:38.678577 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:38.679156 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.680602 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.683090 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.684106 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.684267 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.684509 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:38.684823 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 19:40:38.684986 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1645,"thread_start_us":1446,"threads_started":1}
I20260807 19:40:38.685436 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.675727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:42287 (local address 127.0.0.1:47198)
0807 19:40:38.676355 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:38.676372 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:38.676483 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:38.678556 (+ 2073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:38.678561 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:38.678586 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:38.678948 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:38.678962 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.680748 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.680754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:38.682932 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:38.682942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.684354 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.684360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:38.684577 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.684618 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 19:40:38.684779 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 19:40:38.684971 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":32,"thread_start_us":670,"threads_started":1}
I20260807 19:40:38.685925 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.674439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:33497 (local address 127.0.0.1:45128)
0807 19:40:38.675377 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:38.675416 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:38.675594 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:38.676532 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:38.676543 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:38.676588 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:38.677294 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:38.677309 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.679825 (+ 2516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.679832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:38.682624 (+ 2792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:38.682638 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.684555 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.684562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:38.684732 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.684774 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 19:40:38.684841 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 19:40:38.684983 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":60,"thread_start_us":418,"threads_started":1}
I20260807 19:40:38.685986 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.674439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45128 (local address 127.14.16.65:33497)
0807 19:40:38.675870 (+ 1431us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:38.675882 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:38.675912 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:40:38.676017 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:38.676027 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:38.676263 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:38.677236 (+ 973us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:38.677858 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.679683 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.682787 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.684284 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.685184 (+ 900us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.685433 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:38.685501 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 19:40:38.685659 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1238,"thread_start_us":1030,"threads_started":1}
I20260807 19:40:38.688261 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.676422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:41257 (local address 127.0.0.1:48204)
0807 19:40:38.677305 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:38.677320 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:38.677436 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:38.679885 (+ 2449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:38.679889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:38.679916 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:38.680398 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:38.680408 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.682248 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.682255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:38.684356 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:38.684373 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.687435 (+ 3062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.687442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:38.687612 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.687651 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 19:40:38.687761 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 19:40:38.687888 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":23,"thread_start_us":585,"threads_started":1}
I20260807 19:40:38.688419 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.677106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48204 (local address 127.14.16.67:41257)
0807 19:40:38.679239 (+ 2133us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:38.679248 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:38.679278 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:40:38.679407 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:38.679415 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:38.679654 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:38.679891 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:38.680603 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.682115 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.684538 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.687184 (+ 2646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.687347 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.687569 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:38.687803 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 19:40:38.688035 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1953,"thread_start_us":1800,"threads_started":1}
I20260807 19:40:38.689896 28045 tablet_service.cc:1467] Processing CreateTablet for tablet 968e0b4437314932be1878a35d540c04 (DEFAULT_TABLE table=test-table [id=8ee069c2eea8490bbf0eb0969fca0a77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:40:38.689942 27894 tablet_service.cc:1467] Processing CreateTablet for tablet 968e0b4437314932be1878a35d540c04 (DEFAULT_TABLE table=test-table [id=8ee069c2eea8490bbf0eb0969fca0a77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:40:38.691237 28045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 968e0b4437314932be1878a35d540c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:38.691347 27894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 968e0b4437314932be1878a35d540c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:38.691926 28187 tablet_service.cc:1467] Processing CreateTablet for tablet 968e0b4437314932be1878a35d540c04 (DEFAULT_TABLE table=test-table [id=8ee069c2eea8490bbf0eb0969fca0a77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:40:38.693209 28187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 968e0b4437314932be1878a35d540c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:38.703326 28283 tablet_bootstrap.cc:492] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c: Bootstrap starting.
I20260807 19:40:38.706661 28283 tablet_bootstrap.cc:654] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:38.706645 28284 tablet_bootstrap.cc:492] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038: Bootstrap starting.
I20260807 19:40:38.708122 28283 log.cc:824] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:38.708745 28285 tablet_bootstrap.cc:492] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8: Bootstrap starting.
I20260807 19:40:38.710659 28284 tablet_bootstrap.cc:654] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:38.711777 28285 tablet_bootstrap.cc:654] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:38.712216 28284 log.cc:824] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:38.712801 28285 log.cc:824] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:38.715348 28284 tablet_bootstrap.cc:492] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038: No bootstrap required, opened a new log
I20260807 19:40:38.715764 28284 ts_tablet_manager.cc:1397] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 19:40:38.715868 28285 tablet_bootstrap.cc:492] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8: No bootstrap required, opened a new log
I20260807 19:40:38.716118 28285 ts_tablet_manager.cc:1397] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260807 19:40:38.716269 28283 tablet_bootstrap.cc:492] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c: No bootstrap required, opened a new log
I20260807 19:40:38.716521 28283 ts_tablet_manager.cc:1397] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260807 19:40:38.723246 28284 raft_consensus.cc:348] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0775d1be5ac64423a5fddc0454254038" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 33497 } } peers { permanent_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 42287 } } peers { permanent_uuid: "e7013f40bccb49a1b90088e5e4adb7c8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41257 } }
I20260807 19:40:38.723682 28284 raft_consensus.cc:374] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:38.723793 28284 raft_consensus.cc:729] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0775d1be5ac64423a5fddc0454254038, State: Initialized, Role: FOLLOWER
I20260807 19:40:38.724458 28284 consensus_queue.cc:260] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0775d1be5ac64423a5fddc0454254038" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 33497 } } peers { permanent_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 42287 } } peers { permanent_uuid: "e7013f40bccb49a1b90088e5e4adb7c8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41257 } }
I20260807 19:40:38.725581 27963 heartbeater.cc:503] Master 127.14.16.126:43489 was elected leader, sending a full tablet report...
I20260807 19:40:38.726245 28284 ts_tablet_manager.cc:1428] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260807 19:40:38.727094 28283 raft_consensus.cc:348] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0775d1be5ac64423a5fddc0454254038" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 33497 } } peers { permanent_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 42287 } } peers { permanent_uuid: "e7013f40bccb49a1b90088e5e4adb7c8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41257 } }
I20260807 19:40:38.727094 28285 raft_consensus.cc:348] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0775d1be5ac64423a5fddc0454254038" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 33497 } } peers { permanent_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 42287 } } peers { permanent_uuid: "e7013f40bccb49a1b90088e5e4adb7c8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41257 } }
I20260807 19:40:38.727537 28285 raft_consensus.cc:374] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:38.727537 28283 raft_consensus.cc:374] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:38.727648 28283 raft_consensus.cc:729] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7abd0a2cc3744c11b3738f2e73cb0f4c, State: Initialized, Role: FOLLOWER
I20260807 19:40:38.727674 28285 raft_consensus.cc:729] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7013f40bccb49a1b90088e5e4adb7c8, State: Initialized, Role: FOLLOWER
I20260807 19:40:38.728266 28283 consensus_queue.cc:260] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0775d1be5ac64423a5fddc0454254038" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 33497 } } peers { permanent_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 42287 } } peers { permanent_uuid: "e7013f40bccb49a1b90088e5e4adb7c8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41257 } }
I20260807 19:40:38.728304 28285 consensus_queue.cc:260] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0775d1be5ac64423a5fddc0454254038" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 33497 } } peers { permanent_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 42287 } } peers { permanent_uuid: "e7013f40bccb49a1b90088e5e4adb7c8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41257 } }
I20260807 19:40:38.730656 28285 ts_tablet_manager.cc:1428] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260807 19:40:38.730854 28259 heartbeater.cc:503] Master 127.14.16.126:43489 was elected leader, sending a full tablet report...
I20260807 19:40:38.731016 28283 ts_tablet_manager.cc:1428] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 19:40:38.782034 28290 raft_consensus.cc:482] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:40:38.782385 28290 raft_consensus.cc:504] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0775d1be5ac64423a5fddc0454254038" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 33497 } } peers { permanent_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 42287 } } peers { permanent_uuid: "e7013f40bccb49a1b90088e5e4adb7c8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41257 } }
I20260807 19:40:38.784288 28290 leader_election.cc:288] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7abd0a2cc3744c11b3738f2e73cb0f4c (127.14.16.66:42287), e7013f40bccb49a1b90088e5e4adb7c8 (127.14.16.67:41257)
I20260807 19:40:38.791462 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.785109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:37351 (local address 127.14.16.67:41257)
0807 19:40:38.785335 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:38.785339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:38.785607 (+ 268us) server_negotiation.cc:407] Connection header received
0807 19:40:38.785832 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:38.785836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:38.785901 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:38.785984 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:38.786788 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.787776 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.789639 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.790451 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.790647 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.790729 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:38.791024 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 19:40:38.791160 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 19:40:38.791520 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.784610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:41257 (local address 127.14.16.65:37351)
0807 19:40:38.785434 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:38.785461 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:38.785664 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:38.786038 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:38.786046 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:38.786082 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:38.786601 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:38.786617 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.787905 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.787909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:38.789467 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:38.789481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.790684 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.790691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:38.790860 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.790885 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:40:38.790999 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 19:40:38.791142 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":399,"threads_started":1}
I20260807 19:40:38.793479 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.784637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:57171 (local address 127.14.16.66:42287)
0807 19:40:38.784889 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:38.784896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:38.787962 (+ 3066us) server_negotiation.cc:407] Connection header received
0807 19:40:38.788181 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:38.788186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:38.788280 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:38.788379 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:38.788991 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.789769 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.791513 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.792499 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.792707 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.792780 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:38.793099 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 19:40:38.793229 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:40:38.793557 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.784637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:42287 (local address 127.14.16.65:57171)
0807 19:40:38.787858 (+ 3221us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:38.787881 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:38.788029 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:38.788402 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:38.788407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:38.788433 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:38.788834 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:38.788843 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.789910 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.789916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:38.791359 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:38.791371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.792714 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.792720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:38.792917 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.792954 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:40:38.793072 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 19:40:38.793212 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3084,"thread_start_us":2852,"threads_started":1}
I20260807 19:40:38.793752 28207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968e0b4437314932be1878a35d540c04" candidate_uuid: "0775d1be5ac64423a5fddc0454254038" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7013f40bccb49a1b90088e5e4adb7c8" is_pre_election: true
I20260807 19:40:38.794443 28207 raft_consensus.cc:2454] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0775d1be5ac64423a5fddc0454254038 in term 0.
I20260807 19:40:38.794785 28065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968e0b4437314932be1878a35d540c04" candidate_uuid: "0775d1be5ac64423a5fddc0454254038" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c" is_pre_election: true
I20260807 19:40:38.795264 27843 leader_election.cc:302] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0775d1be5ac64423a5fddc0454254038, e7013f40bccb49a1b90088e5e4adb7c8; no voters:
I20260807 19:40:38.795385 28065 raft_consensus.cc:2454] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0775d1be5ac64423a5fddc0454254038 in term 0.
I20260807 19:40:38.795784 28290 raft_consensus.cc:2781] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:40:38.795959 28290 raft_consensus.cc:482] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:40:38.796082 28290 raft_consensus.cc:3037] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:38.798309 28290 raft_consensus.cc:504] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0775d1be5ac64423a5fddc0454254038" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 33497 } } peers { permanent_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 42287 } } peers { permanent_uuid: "e7013f40bccb49a1b90088e5e4adb7c8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41257 } }
I20260807 19:40:38.799130 28290 leader_election.cc:288] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [CANDIDATE]: Term 1 election: Requested vote from peers 7abd0a2cc3744c11b3738f2e73cb0f4c (127.14.16.66:42287), e7013f40bccb49a1b90088e5e4adb7c8 (127.14.16.67:41257)
I20260807 19:40:38.799660 28065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968e0b4437314932be1878a35d540c04" candidate_uuid: "0775d1be5ac64423a5fddc0454254038" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c"
I20260807 19:40:38.799700 28207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "968e0b4437314932be1878a35d540c04" candidate_uuid: "0775d1be5ac64423a5fddc0454254038" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7013f40bccb49a1b90088e5e4adb7c8"
I20260807 19:40:38.799888 28065 raft_consensus.cc:3037] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:38.799952 28207 raft_consensus.cc:3037] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:38.802590 28065 raft_consensus.cc:2454] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0775d1be5ac64423a5fddc0454254038 in term 1.
I20260807 19:40:38.802637 28207 raft_consensus.cc:2454] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0775d1be5ac64423a5fddc0454254038 in term 1.
I20260807 19:40:38.802994 27845 leader_election.cc:302] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0775d1be5ac64423a5fddc0454254038, 7abd0a2cc3744c11b3738f2e73cb0f4c; no voters:
I20260807 19:40:38.803411 28290 raft_consensus.cc:2781] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:40:38.803876 28290 raft_consensus.cc:686] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 1 LEADER]: Becoming Leader. State: Replica: 0775d1be5ac64423a5fddc0454254038, State: Running, Role: LEADER
I20260807 19:40:38.804368 28290 consensus_queue.cc:237] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0775d1be5ac64423a5fddc0454254038" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 33497 } } peers { permanent_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 42287 } } peers { permanent_uuid: "e7013f40bccb49a1b90088e5e4adb7c8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41257 } }
I20260807 19:40:38.808749 27753 catalog_manager.cc:5495] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0775d1be5ac64423a5fddc0454254038 (127.14.16.65). New cstate: current_term: 1 leader_uuid: "0775d1be5ac64423a5fddc0454254038" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0775d1be5ac64423a5fddc0454254038" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 33497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 42287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7013f40bccb49a1b90088e5e4adb7c8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41257 } health_report { overall_health: UNKNOWN } } }
W20260807 19:40:38.830996 28117 tablet.cc:2369] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:40:38.832446 28261 tablet.cc:2369] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:40:38.845348 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.839998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45134 (local address 127.14.16.65:33497)
0807 19:40:38.840258 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:38.840266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:38.840318 (+ 52us) server_negotiation.cc:407] Connection header received
0807 19:40:38.840603 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:38.840608 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:38.840698 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:38.840825 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:40:38.841153 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.841910 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.843354 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.843766 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.843861 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.844352 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:40:38.844842 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:40:38.844953 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:38.845058 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 19:40:38.845123 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:40:38.853762 28065 raft_consensus.cc:1261] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c [term 1 FOLLOWER]: Refusing update from remote peer 0775d1be5ac64423a5fddc0454254038: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:40:38.853818 28207 raft_consensus.cc:1261] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8 [term 1 FOLLOWER]: Refusing update from remote peer 0775d1be5ac64423a5fddc0454254038: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:40:38.854835 28295 consensus_queue.cc:1035] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7013f40bccb49a1b90088e5e4adb7c8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:40:38.855268 28290 consensus_queue.cc:1035] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7abd0a2cc3744c11b3738f2e73cb0f4c" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 42287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:40:38.860333 27964 tablet.cc:2369] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:40:38.878134 28300 mvcc.cc:205] Tried to move back new op lower bound from 7315995192734318592 to 7315995192546226176. Current Snapshot: MvccSnapshot[applied={T|T < 7315995192734318592}]
I20260807 19:40:38.881050 28299 mvcc.cc:205] Tried to move back new op lower bound from 7315995192734318592 to 7315995192546226176. Current Snapshot: MvccSnapshot[applied={T|T < 7315995192734318592 or (T in {7315995192734318592})}]
I20260807 19:40:38.896368 27752 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:40:38.896678 27752 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:40:38.900785 27752 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad20dfd3473b436893258cfd36f30695: Sending DeleteTablet for 3 replicas of tablet 968e0b4437314932be1878a35d540c04
I20260807 19:40:38.902024 27894 tablet_service.cc:1514] Processing DeleteTablet for tablet 968e0b4437314932be1878a35d540c04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:40:38 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:45128
I20260807 19:40:38.902029 28187 tablet_service.cc:1514] Processing DeleteTablet for tablet 968e0b4437314932be1878a35d540c04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:40:38 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:48204
I20260807 19:40:38.902069 28045 tablet_service.cc:1514] Processing DeleteTablet for tablet 968e0b4437314932be1878a35d540c04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:40:38 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:47198
I20260807 19:40:38.908296 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 27795
I20260807 19:40:38.928558 28309 tablet_replica.cc:330] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8: stopping tablet replica
I20260807 19:40:38.929069 28310 tablet_replica.cc:330] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038: stopping tablet replica
I20260807 19:40:38.929633 28310 raft_consensus.cc:2229] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:40:38.930202 28309 raft_consensus.cc:2229] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:40:38.930835 28309 raft_consensus.cc:2258] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:40:38.931802 28310 raft_consensus.cc:2258] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:40:38.934672 28310 ts_tablet_manager.cc:1905] T 968e0b4437314932be1878a35d540c04 P 0775d1be5ac64423a5fddc0454254038: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:40:38.936559 28309 ts_tablet_manager.cc:1905] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:40:38.937294 28311 tablet_replica.cc:330] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c: stopping tablet replica
I20260807 19:40:38.937789 28311 raft_consensus.cc:2229] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:40:38.938333 28311 raft_consensus.cc:2258] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:40:38.939863 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:38.909790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45144 (local address 127.14.16.65:33497)
0807 19:40:38.910375 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:38.910381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:38.910398 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:40:38.910456 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:38.910465 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:38.910575 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:38.910682 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:38.917898 (+ 7216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.919522 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.922158 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:38.923123 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:38.923271 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:38.928327 (+ 5056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:38.928343 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:38.928365 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:38.928439 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:38.933852 (+ 5413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:38.934866 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:38.934875 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:38.934878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:38.935069 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:38.936750 (+ 1681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:38.936756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:38.936759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:38.937230 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:38.938973 (+ 1743us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:38.939158 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 19:40:38.939465 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454}
I20260807 19:40:38.940794 28311 ts_tablet_manager.cc:1905] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:40:38.940851 27958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:38.948594 28309 ts_tablet_manager.cc:1918] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:40:38.948794 28309 log.cc:1196] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/968e0b4437314932be1878a35d540c04
I20260807 19:40:38.949172 28311 ts_tablet_manager.cc:1918] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:40:38.949345 28309 ts_tablet_manager.cc:1939] T 968e0b4437314932be1878a35d540c04 P e7013f40bccb49a1b90088e5e4adb7c8: Deleting consensus metadata
I20260807 19:40:38.949353 28311 log.cc:1196] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/968e0b4437314932be1878a35d540c04
I20260807 19:40:38.949920 28311 ts_tablet_manager.cc:1939] T 968e0b4437314932be1878a35d540c04 P 7abd0a2cc3744c11b3738f2e73cb0f4c: Deleting consensus metadata
I20260807 19:40:38.951155 27737 catalog_manager.cc:4841] TS e7013f40bccb49a1b90088e5e4adb7c8 (127.14.16.67:41257): tablet 968e0b4437314932be1878a35d540c04 (table test-table [id=8ee069c2eea8490bbf0eb0969fca0a77]) successfully deleted
I20260807 19:40:38.951694 27739 catalog_manager.cc:4841] TS 7abd0a2cc3744c11b3738f2e73cb0f4c (127.14.16.66:42287): tablet 968e0b4437314932be1878a35d540c04 (table test-table [id=8ee069c2eea8490bbf0eb0969fca0a77]) successfully deleted
I20260807 19:40:39.042433 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 27795
W20260807 19:40:39.062726 27739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.16.65:33497 (error 108)
I20260807 19:40:39.062857 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 27967
I20260807 19:40:39.064843 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:39.064350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:33497 (local address 127.0.0.1:45150)
0807 19:40:39.064541 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:39.064666 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 19:40:39.065125 27739 catalog_manager.cc:4584] TS 0775d1be5ac64423a5fddc0454254038 (127.14.16.65:33497): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 968e0b4437314932be1878a35d540c04: Network error: Client connection negotiation failed: client connection to 127.14.16.65:33497: connect: Connection refused (error 111)
I20260807 19:40:39.076820 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:39.063818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47202 (local address 127.14.16.66:42287)
0807 19:40:39.063993 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:39.063999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:39.064044 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:40:39.064331 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:39.064335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:39.064435 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:39.064557 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:39.066019 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:39.066726 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:39.068159 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:39.068564 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:39.068656 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:39.071217 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:39.071236 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:39.071250 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:39.071277 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:39.074043 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:39.074763 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:39.074770 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:39.074772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:39.074896 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:39.075414 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:39.075417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:39.075419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:39.075769 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:39.076018 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:39.076286 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 19:40:39.076445 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:40:39.077543 28109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:39.118646 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:39.118098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:33497 (local address 127.0.0.1:45158)
0807 19:40:39.118333 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:39.118461 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 19:40:39.119735 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:39.119354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:33497 (local address 127.0.0.1:45170)
0807 19:40:39.119503 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:39.119604 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 19:40:39.169150 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 27967
I20260807 19:40:39.180437 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:39.179943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:33497 (local address 127.0.0.1:45174)
0807 19:40:39.180167 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:39.180263 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 19:40:39.181429 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:39.181099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:33497 (local address 127.0.0.1:45178)
0807 19:40:39.181245 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:39.181321 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 19:40:39.188881 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 28120
I20260807 19:40:39.202349 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:39.189622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48208 (local address 127.14.16.67:41257)
0807 19:40:39.189839 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:39.189845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:39.189867 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:40:39.190000 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:39.190004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:39.190087 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:39.190185 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:39.191699 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:39.192422 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:39.193702 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:39.194124 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:39.194236 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:39.196875 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:39.196890 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:39.196906 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:39.196941 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:39.199828 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:39.200481 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:39.200488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:39.200491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:39.200611 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:39.200963 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:39.200966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:39.200968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:39.201308 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:39.201552 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:39.201834 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 19:40:39.202001 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 19:40:39.203119 28249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:39.262163 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:39.261601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:33497 (local address 127.0.0.1:45188)
0807 19:40:39.261845 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:39.261967 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 19:40:39.263585 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:39.263055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:33497 (local address 127.0.0.1:45198)
0807 19:40:39.263330 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:39.263422 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260807 19:40:39.293929 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 28120
I20260807 19:40:39.312278 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 27720
I20260807 19:40:39.313621 27788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:39.429240 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 27720
I20260807 19:40:39.453871 14401 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 19:40:39.799455 14401 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 19:40:39.799728 14401 mini_ranger.cc:66] Stopping Ranger...
I20260807 19:40:40.144886 14401 mini_ranger.cc:68] Stopped Ranger
2026-08-07 19:40:40.145 UTC [26735] LOG: received smart shutdown request
2026-08-07 19:40:40.148 UTC [26735] LOG: background worker "logical replication launcher" (PID 26746) exited with exit code 1
2026-08-07 19:40:40.148 UTC [26739] LOG: shutting down
2026-08-07 19:40:40.311 UTC [26735] LOG: database system is shut down
2026-08-07T19:40:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54776 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:40:40 dist-test-slave-07c2 krb5kdc[28325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:40:40 dist-test-slave-07c2 krb5kdc[28325](info): set up 2 sockets
Aug 07 19:40:40 dist-test-slave-07c2 krb5kdc[28325](info): commencing operation
krb5kdc: starting...
W20260807 19:40:41.615079 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:40:41 dist-test-slave-07c2 krb5kdc[28325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:40:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:40:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:40:41.891047 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:42905
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:42905,127.14.16.125:36345,127.14.16.124:46625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:40:42.230818 28338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:42.231112 28338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:42.231241 28338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:42.238759 28338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:40:42.238873 28338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:42.238976 28338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:42.239085 28338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:40:42.239169 28338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:40:42.249372 28338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:42905,127.14.16.125:36345,127.14.16.124:46625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:42905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:42.250835 28338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:42.252260 28338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:42.265409 28344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:42.265380 28345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:42.265350 28347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:42.266335 28338 server_base.cc:1034] running on GCE node
I20260807 19:40:42.267146 28338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:42.268729 28338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:42.269969 28338 hybrid_clock.cc:648] HybridClock initialized: now 1786131642269935 us; error 52 us; skew 500 ppm
I20260807 19:40:42.270327 28338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:42 dist-test-slave-07c2 krb5kdc[28325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:42.278730 28338 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:40:42.280418 28338 webserver.cc:460] Webserver started at http://127.14.16.126:43823/ using document root <none> and password file <none>
I20260807 19:40:42.281170 28338 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:42.281325 28338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:42.281670 28338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:40:42.284842 28338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "db42b8af284d49978d59abbf78ceaa38"
format_stamp: "Formatted at 2026-08-07 19:40:42 on dist-test-slave-07c2"
server_key: "6977c59616a76e9de8b9ab3a4b93a38d"
server_key_iv: "87246050afa91124c3554a16986df7bf"
server_key_version: "encryptionkey@0"
I20260807 19:40:42.285699 28338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "db42b8af284d49978d59abbf78ceaa38"
format_stamp: "Formatted at 2026-08-07 19:40:42 on dist-test-slave-07c2"
server_key: "6977c59616a76e9de8b9ab3a4b93a38d"
server_key_iv: "87246050afa91124c3554a16986df7bf"
server_key_version: "encryptionkey@0"
I20260807 19:40:42.291231 28338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:40:42.294838 28354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:42.296149 28338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:40:42.296344 28338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "db42b8af284d49978d59abbf78ceaa38"
format_stamp: "Formatted at 2026-08-07 19:40:42 on dist-test-slave-07c2"
server_key: "6977c59616a76e9de8b9ab3a4b93a38d"
server_key_iv: "87246050afa91124c3554a16986df7bf"
server_key_version: "encryptionkey@0"
I20260807 19:40:42.296546 28338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:42.321798 28338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:42.325928 28338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:42.326241 28338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:42.345155 28338 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:42905
I20260807 19:40:42.345196 28407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:42905 every 8 connection(s)
I20260807 19:40:42.346527 28338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:40:42.349586 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 28338
I20260807 19:40:42.349828 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:40:42.350257 28408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:42.350558 14401 external_mini_cluster.cc:1428] Setting key 435defbc3c8d44b7c293811061b989a7
I20260807 19:40:42.356678 28408 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:40:42 dist-test-slave-07c2 krb5kdc[28325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.16.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.125@KRBTEST.COM" created.
I20260807 19:40:42.382403 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.359709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:42905 (local address 127.0.0.1:59124)
0807 19:40:42.362238 (+ 2529us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.362294 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:42.364400 (+ 2106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:42.365382 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:42.365387 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:42.365930 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:42.366279 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:42.366296 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:42.368701 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:42.368704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:42.369771 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:42.369783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:42.371151 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:42.371157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:42.371412 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:42.372095 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:42.372118 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:42.374276 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:42.378507 (+ 4231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:42.378518 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:42.378526 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:42.379042 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:42.379526 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:42.379530 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:42.379532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:42.379684 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:42.380538 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:42.380548 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:42.380774 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 19:40:42.380987 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 19:40:42.381377 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2237,"thread_start_us":2043,"threads_started":1}
I20260807 19:40:42.382433 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.362702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.14.16.126:42905)
0807 19:40:42.363463 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:42.363473 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:42.363529 (+ 56us) server_negotiation.cc:407] Connection header received
0807 19:40:42.364789 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:42.364799 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:42.365113 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:42.365372 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:42.366477 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:42.368525 (+ 2048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:42.369917 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:42.370872 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:42.371059 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:42.374543 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:42.374555 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:42.374574 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:42.374627 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:42.378373 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:42.379183 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:42.379190 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:42.379192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:42.379423 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:42.379845 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:42.379849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:42.379851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:42.380353 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:42.380582 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:42.381182 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 19:40:42.381583 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":504,"thread_start_us":324,"threads_started":1}
I20260807 19:40:42.385758 28408 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:40:42.386931 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.386537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54318)
0807 19:40:42.386712 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.386819 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 19:40:42.387274 28358 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
I20260807 19:40:42.388931 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.388584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54322)
0807 19:40:42.388742 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.388809 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 19:40:42.389230 28408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:36345: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.125.keytab.
Entry for principal kudu/127.14.16.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.125.keytab.
I20260807 19:40:42.412736 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.125:36345
--webserver_interface=127.14.16.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.125.keytab
--principal=kudu/127.14.16.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:42905,127.14.16.125:36345,127.14.16.124:46625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:40:42.438621 28408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } attempt: 1
I20260807 19:40:42.440042 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.439469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54324)
0807 19:40:42.439730 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.439859 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 19:40:42.441067 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.440689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54340)
0807 19:40:42.440835 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.440916 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 19:40:42.441273 28408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:36345: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
I20260807 19:40:42.509531 28408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } attempt: 2
I20260807 19:40:42.510908 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.510271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54344)
0807 19:40:42.510585 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.510713 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 19:40:42.511984 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.511573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54354)
0807 19:40:42.511723 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.511822 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 19:40:42.512185 28408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:36345: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
I20260807 19:40:42.603497 28408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } attempt: 3
I20260807 19:40:42.604873 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.604301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54366)
0807 19:40:42.604550 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.604685 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 19:40:42.606168 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.605707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54380)
0807 19:40:42.605877 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.605985 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 19:40:42.606400 28408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:36345: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
I20260807 19:40:42.749662 28408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } attempt: 4
I20260807 19:40:42.750993 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.750342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54382)
0807 19:40:42.750709 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.750822 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
I20260807 19:40:42.752132 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.751720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54394)
0807 19:40:42.751874 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.751976 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 19:40:42.752404 28408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:36345: Network error: Client connection negotiation failed: client connection to 127.14.16.125:36345: connect: Connection refused (error 111)
W20260807 19:40:42.755255 28415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:42.755620 28415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:42.755765 28415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:42.763024 28415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:40:42.763151 28415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:42.763257 28415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:42.763392 28415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:40:42.763484 28415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:40:42.773751 28415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:42905,127.14.16.125:36345,127.14.16.124:46625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.125.keytab
--principal=kudu/127.14.16.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.125:36345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.16.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:42.775092 28415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:42.776451 28415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:42.788322 28420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:42.788254 28421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:42.788480 28423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:42.789274 28415 server_base.cc:1034] running on GCE node
I20260807 19:40:42.790040 28415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:42.791440 28415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:42.792676 28415 hybrid_clock.cc:648] HybridClock initialized: now 1786131642792634 us; error 49 us; skew 500 ppm
I20260807 19:40:42.793025 28415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:42 dist-test-slave-07c2 krb5kdc[28325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:42.797348 28415 init.cc:377] Logged in from keytab as kudu/127.14.16.125@KRBTEST.COM (short username kudu)
I20260807 19:40:42.799010 28415 webserver.cc:460] Webserver started at http://127.14.16.125:45629/ using document root <none> and password file <none>
I20260807 19:40:42.799801 28415 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:42.799957 28415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:42.800299 28415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:40:42.803376 28415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/data/instance:
uuid: "6cebf5b997f9446e99e539bcb514ba09"
format_stamp: "Formatted at 2026-08-07 19:40:42 on dist-test-slave-07c2"
server_key: "55a7bf39db65d1463e78d2aef11cba35"
server_key_iv: "044a34f35edd67dcc4797129ca093d1f"
server_key_version: "encryptionkey@0"
I20260807 19:40:42.804200 28415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/wal/instance:
uuid: "6cebf5b997f9446e99e539bcb514ba09"
format_stamp: "Formatted at 2026-08-07 19:40:42 on dist-test-slave-07c2"
server_key: "55a7bf39db65d1463e78d2aef11cba35"
server_key_iv: "044a34f35edd67dcc4797129ca093d1f"
server_key_version: "encryptionkey@0"
I20260807 19:40:42.809263 28415 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:40:42.812618 28430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:42.813841 28415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:40:42.814023 28415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/wal
uuid: "6cebf5b997f9446e99e539bcb514ba09"
format_stamp: "Formatted at 2026-08-07 19:40:42 on dist-test-slave-07c2"
server_key: "55a7bf39db65d1463e78d2aef11cba35"
server_key_iv: "044a34f35edd67dcc4797129ca093d1f"
server_key_version: "encryptionkey@0"
I20260807 19:40:42.814213 28415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:42.836130 28415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:42.840718 28415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:42.841022 28415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:42.860306 28415 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.125:36345
I20260807 19:40:42.860345 28484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.125:36345 every 8 connection(s)
I20260807 19:40:42.861644 28415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
I20260807 19:40:42.865222 28485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:42.870301 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 28415
I20260807 19:40:42.870519 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-1/wal/instance
I20260807 19:40:42.871014 14401 external_mini_cluster.cc:1428] Setting key 7f8d9513f14ffb6c1452f884db36901f
I20260807 19:40:42.871716 28485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:40:42 dist-test-slave-07c2 krb5kdc[28325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.125@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:42.892472 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.874288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59140 (local address 127.14.16.126:42905)
0807 19:40:42.874533 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:42.874542 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:42.875695 (+ 1153us) server_negotiation.cc:407] Connection header received
0807 19:40:42.877607 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:42.877615 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:42.877713 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:42.877837 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:42.879350 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:42.880416 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:42.882044 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:42.882468 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:42.882630 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:42.886243 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:42.886248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:42.886252 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:42.886287 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:42.889273 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:42.890030 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:42.890035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:42.890039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:42.890178 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:42.890612 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:42.890618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:42.890621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:42.890922 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:42.891117 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:42.891885 (+ 768us) server_negotiation.cc:299] Negotiation successful
0807 19:40:42.892131 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:40:42.893501 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.874682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:42905 (local address 127.0.0.1:59140)
0807 19:40:42.875512 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.875567 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:42.877323 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:42.878029 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:42.878037 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:42.878553 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:42.879104 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:42.879120 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:42.880598 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:42.880602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:42.881885 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:42.881898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:42.882728 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:42.882736 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:42.882946 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:42.883756 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:42.883785 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:42.885994 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:42.889424 (+ 3430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:42.889435 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:42.889453 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:42.889904 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:42.890293 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:42.890298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:42.890302 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:42.890491 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:42.891076 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:42.891085 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:42.891518 (+ 433us) client_negotiation.cc:769] Sending connection context
0807 19:40:42.891804 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 19:40:42.892344 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":365,"threads_started":1}
I20260807 19:40:42.895577 28485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.14.16.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.124@KRBTEST.COM" created.
Aug 07 19:40:42 dist-test-slave-07c2 krb5kdc[28325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.125@KRBTEST.COM for kudu/127.14.16.125@KRBTEST.COM
I20260807 19:40:42.912063 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.896450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54410)
0807 19:40:42.896665 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.896684 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:42.896795 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:42.897901 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:42.897904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:42.898271 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:42.898409 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:42.898417 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:42.900410 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:42.900418 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:42.902041 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:42.902061 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:42.902910 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:42.902918 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:42.903156 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:42.903804 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:42.903832 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:42.906316 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:42.909519 (+ 3203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:42.909527 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:42.909531 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:42.909890 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:42.910243 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:42.910249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:42.910252 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:42.910400 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:42.910992 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:42.910999 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:42.911170 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 19:40:42.911347 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 19:40:42.911620 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 19:40:42.912139 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.896712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54410 (local address 127.14.16.125:36345)
0807 19:40:42.897433 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:42.897439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:42.897464 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:40:42.897507 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:42.897512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:42.897751 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:42.897870 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:42.898583 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:42.900269 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:42.902202 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:42.902744 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:42.902848 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:42.906520 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:42.906531 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:42.906544 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:42.906575 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:42.909397 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:42.910016 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:42.910031 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:42.910033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:42.910140 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:42.910513 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:42.910520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:42.910523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:42.910861 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:42.911089 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:42.911490 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 19:40:42.911745 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":576,"thread_start_us":432,"threads_started":1}
I20260807 19:40:42.914393 28485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:40:42.915620 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.915109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42568)
0807 19:40:42.915360 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.915473 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 19:40:42.915975 28431 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
I20260807 19:40:42.917620 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.917265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42582)
0807 19:40:42.917443 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.917519 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 19:40:42.917950 28485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46625: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.124.keytab.
Entry for principal kudu/127.14.16.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.124.keytab.
I20260807 19:40:42.933230 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.124:46625
--webserver_interface=127.14.16.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.124.keytab
--principal=kudu/127.14.16.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:36667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:42905,127.14.16.125:36345,127.14.16.124:46625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:40:42.967335 28485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } attempt: 1
I20260807 19:40:42.968552 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.968091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42586)
0807 19:40:42.968285 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.968388 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 19:40:42.969503 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:42.969162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42602)
0807 19:40:42.969330 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:42.969397 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 19:40:42.969712 28485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46625: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
I20260807 19:40:43.037921 28485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } attempt: 2
I20260807 19:40:43.039083 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.038614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42614)
0807 19:40:43.038823 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.038922 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 19:40:43.040220 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.039891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42622)
0807 19:40:43.040038 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.040111 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 19:40:43.040429 28485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46625: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
I20260807 19:40:43.051656 28408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } attempt: 5
Aug 07 19:40:43 dist-test-slave-07c2 krb5kdc[28325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for kudu/127.14.16.125@KRBTEST.COM
I20260807 19:40:43.066169 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.052326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54426)
0807 19:40:43.052553 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.052576 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:43.052711 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:43.053067 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:43.053072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:43.053591 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:43.053834 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:43.053846 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.054847 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.054852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:43.056258 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:43.056276 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.057033 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.057040 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:43.057248 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.057919 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:43.057941 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:43.060547 (+ 2606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:43.063710 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:43.063716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:43.063720 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:43.064093 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:43.064470 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:43.064475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:43.064478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:43.064622 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:43.065165 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:43.065170 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:43.065337 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 19:40:43.065483 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 19:40:43.065782 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 19:40:43.066154 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.052440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54426 (local address 127.14.16.125:36345)
0807 19:40:43.052605 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:43.052609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:43.052631 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:40:43.052851 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:43.052855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:43.052916 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:43.052999 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:43.054004 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.054701 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.056407 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.056874 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.056974 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.060755 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:43.060760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:43.060763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:43.060796 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:43.063567 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.064215 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.064222 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.064225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.064351 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.064745 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.064750 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.064753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.065041 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:43.065200 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:43.065594 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 19:40:43.065819 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:40:43.067260 28408 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:40:43.068356 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.067987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42628)
0807 19:40:43.068147 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.068220 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:40:43.069254 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.068940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42636)
0807 19:40:43.069073 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.069158 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 19:40:43.069481 28408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46625: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
I20260807 19:40:43.120694 28408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } attempt: 1
I20260807 19:40:43.121927 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.121455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42642)
0807 19:40:43.121655 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.121753 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 19:40:43.122970 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.122607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42652)
0807 19:40:43.122776 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.122867 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 19:40:43.123183 28408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46625: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
I20260807 19:40:43.131626 28485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } attempt: 3
I20260807 19:40:43.132751 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.132305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42662)
0807 19:40:43.132508 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.132590 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 19:40:43.133778 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.133377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42664)
0807 19:40:43.133598 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.133671 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260807 19:40:43.133980 28485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46625: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
I20260807 19:40:43.191430 28408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } attempt: 2
I20260807 19:40:43.192639 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.192128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42670)
0807 19:40:43.192378 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.192466 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 19:40:43.193639 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.193305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42686)
0807 19:40:43.193453 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.193539 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 19:40:43.193857 28408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46625: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
I20260807 19:40:43.277179 28485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } attempt: 4
I20260807 19:40:43.278374 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.277835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42688)
0807 19:40:43.278081 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.278192 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:40:43.279392 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.279015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42702)
0807 19:40:43.279165 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.279238 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 19:40:43.279601 28485 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46625: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
W20260807 19:40:43.281132 28492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:43.281466 28492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:43.281654 28492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:43.289143 28492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:40:43.289300 28492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:43.289395 28492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:43.289505 28492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:40:43.289587 28492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:40:43.299788 28492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:36667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:42905,127.14.16.125:36345,127.14.16.124:46625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.124.keytab
--principal=kudu/127.14.16.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.124:46625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.16.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:43.300084 28408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } attempt: 3
I20260807 19:40:43.301275 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.300801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42710)
0807 19:40:43.301014 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.301100 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 19:40:43.301610 28492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:43.302359 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.301956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42712)
0807 19:40:43.302164 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.302252 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260807 19:40:43.302596 28408 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46625: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46625: connect: Connection refused (error 111)
I20260807 19:40:43.303189 28492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:43.315150 28497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:43.315265 28500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:43.315230 28498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:43.315907 28492 server_base.cc:1034] running on GCE node
I20260807 19:40:43.316672 28492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:43.318027 28492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:43.319267 28492 hybrid_clock.cc:648] HybridClock initialized: now 1786131643319221 us; error 39 us; skew 500 ppm
I20260807 19:40:43.319618 28492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:43 dist-test-slave-07c2 krb5kdc[28325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131643, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:43.322692 28492 init.cc:377] Logged in from keytab as kudu/127.14.16.124@KRBTEST.COM (short username kudu)
I20260807 19:40:43.324258 28492 webserver.cc:460] Webserver started at http://127.14.16.124:42223/ using document root <none> and password file <none>
I20260807 19:40:43.324939 28492 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:43.325071 28492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:43.325394 28492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:40:43.328459 28492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/data/instance:
uuid: "3d70c92c32c14225958a6819c6911de2"
format_stamp: "Formatted at 2026-08-07 19:40:43 on dist-test-slave-07c2"
server_key: "36d1ec3d4b54a403386bce2eee20c1f5"
server_key_iv: "b4d46862c2cf9e16065b53de3614bf9c"
server_key_version: "encryptionkey@0"
I20260807 19:40:43.329305 28492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/wal/instance:
uuid: "3d70c92c32c14225958a6819c6911de2"
format_stamp: "Formatted at 2026-08-07 19:40:43 on dist-test-slave-07c2"
server_key: "36d1ec3d4b54a403386bce2eee20c1f5"
server_key_iv: "b4d46862c2cf9e16065b53de3614bf9c"
server_key_version: "encryptionkey@0"
I20260807 19:40:43.334259 28492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 19:40:43.337455 28507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:43.338902 28492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:40:43.339076 28492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/wal
uuid: "3d70c92c32c14225958a6819c6911de2"
format_stamp: "Formatted at 2026-08-07 19:40:43 on dist-test-slave-07c2"
server_key: "36d1ec3d4b54a403386bce2eee20c1f5"
server_key_iv: "b4d46862c2cf9e16065b53de3614bf9c"
server_key_version: "encryptionkey@0"
I20260807 19:40:43.339324 28492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:43.361979 28492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:43.366333 28492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:43.366643 28492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:43.386057 28492 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.124:46625
I20260807 19:40:43.386149 28561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.124:46625 every 8 connection(s)
I20260807 19:40:43.387563 28492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
I20260807 19:40:43.391366 28562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:43.392047 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 28492
I20260807 19:40:43.392256 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131435615921-14401-0/minicluster-data/master-2/wal/instance
I20260807 19:40:43.392781 14401 external_mini_cluster.cc:1428] Setting key 1cfbc617617e8e291241e404c40aebdf
I20260807 19:40:43.401342 28562 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:40:43 dist-test-slave-07c2 krb5kdc[28325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.124@KRBTEST.COM
Aug 07 19:40:43 dist-test-slave-07c2 krb5kdc[28325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131643, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.124@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:43.417397 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.394754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.14.16.124:46625)
0807 19:40:43.395695 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:43.395710 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:43.395762 (+ 52us) server_negotiation.cc:407] Connection header received
0807 19:40:43.398159 (+ 2397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:43.398188 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:43.398720 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:43.399177 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:43.400903 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.402862 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.404663 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.405668 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.405889 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.410015 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:43.410023 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:43.410034 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:43.410073 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:43.413366 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.414069 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.414076 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.414081 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.414229 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.414617 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.414621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.414623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.415118 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:43.415473 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:43.415913 (+ 440us) server_negotiation.cc:299] Negotiation successful
0807 19:40:43.416305 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":446,"threads_started":1}
I20260807 19:40:43.419112 28566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.404256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59154 (local address 127.14.16.126:42905)
0807 19:40:43.404863 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:43.404870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:43.405291 (+ 421us) server_negotiation.cc:407] Connection header received
0807 19:40:43.405547 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:43.405551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:43.405630 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:43.405744 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:43.406854 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.407675 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.408959 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.409458 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.409612 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.413093 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:43.413100 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:43.413104 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:43.413148 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:43.416010 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.416887 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.416895 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.416900 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.417034 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.417438 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.417444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.417448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.417872 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:43.418036 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:43.418472 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 19:40:43.418699 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":295,"threads_started":1}
I20260807 19:40:43.419131 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.404309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:42905 (local address 127.0.0.1:59154)
0807 19:40:43.405158 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.405191 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:43.405383 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:43.405807 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:43.405814 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:43.406441 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:43.406701 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:43.406710 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.407824 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.407831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:43.408815 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:43.408825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.409699 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.409707 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:43.409891 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.410572 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:43.410590 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:43.412887 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:43.416328 (+ 3441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:43.416342 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:43.416349 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:43.416768 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:43.417155 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:43.417160 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:43.417164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:43.417313 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:43.418002 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:43.418010 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:43.418188 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 19:40:43.418356 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 19:40:43.418670 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":476,"threads_started":1}
I20260807 19:40:43.420603 28562 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:40:43 dist-test-slave-07c2 krb5kdc[28325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131643, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.124@KRBTEST.COM for kudu/127.14.16.125@KRBTEST.COM
Aug 07 19:40:43 dist-test-slave-07c2 krb5kdc[28325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:43.434234 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.421394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:36345 (local address 127.0.0.1:54434)
0807 19:40:43.421616 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.421632 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:43.421758 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:43.422070 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:43.422074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:43.422436 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:43.422567 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:43.422575 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.423613 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.423619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:43.424486 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:43.424497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.425203 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.425209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:43.425423 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.425918 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:43.425934 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:43.427867 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:43.431747 (+ 3880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:43.431751 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:43.431754 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:43.432119 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:43.432546 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:43.432550 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:43.432552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:43.432707 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:43.433265 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:43.433270 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:43.433434 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 19:40:43.433577 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 19:40:43.433867 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260807 19:40:43.434273 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.421389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54434 (local address 127.14.16.125:36345)
0807 19:40:43.421569 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:43.421575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:43.421709 (+ 134us) server_negotiation.cc:407] Connection header received
0807 19:40:43.421870 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:43.421875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:43.421948 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:43.422050 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:43.422708 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.423479 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.424626 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.425053 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.425184 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.428061 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:43.428067 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:43.428072 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:43.428116 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:43.431611 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.432258 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.432264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.432267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.432431 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.432817 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.432822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.432824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.433135 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:43.433311 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:43.433668 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 19:40:43.433872 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:40:43.435421 28562 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:40:43.435822 28566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.420584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59156 (local address 127.14.16.126:42905)
0807 19:40:43.421092 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:43.421096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:43.421109 (+ 13us) server_negotiation.cc:407] Connection header received
0807 19:40:43.421173 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:43.421177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:43.421266 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:43.421382 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:43.422633 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.423380 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.424633 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.425067 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.425200 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.429614 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:43.429620 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:43.429626 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:43.429670 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:43.432865 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.433492 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.433499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.433502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.433633 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.434000 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.434005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.434008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.434286 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:43.434602 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:43.435253 (+ 651us) server_negotiation.cc:299] Negotiation successful
0807 19:40:43.435531 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391}
Aug 07 19:40:43 dist-test-slave-07c2 krb5kdc[28325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131643, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.124@KRBTEST.COM for kudu/127.14.16.124@KRBTEST.COM
Aug 07 19:40:43 dist-test-slave-07c2 krb5kdc[28325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.125@KRBTEST.COM
I20260807 19:40:43.449007 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.436229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.14.16.124:46625)
0807 19:40:43.436383 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:43.436389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:43.436414 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:40:43.436581 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:43.436586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:43.436663 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:43.436767 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:43.437823 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.438631 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.439825 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.440338 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.440469 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.443499 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:43.443506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:43.443511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:43.443545 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:43.446326 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.446957 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.446963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.446967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.447114 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.447547 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.447551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.447553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.447820 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:43.447974 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:43.448380 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 19:40:43.448734 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":197,"server-negotiator.queue_time_us":52}
I20260807 19:40:43.449095 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.436172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42724)
0807 19:40:43.436353 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.436373 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:43.436471 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:43.436799 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:43.436802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:43.437305 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:43.437466 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:43.437474 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.438760 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.438767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:43.439690 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:43.439702 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.440476 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.440481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:43.440726 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.441289 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:43.441304 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:43.443260 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:43.446449 (+ 3189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:43.446452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:43.446455 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:43.446832 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:43.447230 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:43.447233 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:43.447235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:43.447435 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:43.447951 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:43.447955 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:43.448108 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 19:40:43.448274 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 19:40:43.448608 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":25}
I20260807 19:40:43.453524 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.437674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54438 (local address 127.14.16.125:36345)
0807 19:40:43.437847 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:43.437852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:43.437877 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:40:43.438150 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:43.438154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:43.438209 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:43.438309 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:43.439548 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.440367 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.441841 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.442262 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.442481 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.447400 (+ 4919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:43.447406 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:43.447410 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:43.447446 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:43.450867 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.451714 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.451717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.451720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.451849 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.452257 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.452260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.452262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.452515 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:43.452717 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:43.453095 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 19:40:43.453259 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:40:43.454942 28562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2: Bootstrap starting.
I20260807 19:40:43.458149 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:40:43.458667 28562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:43.459827 28562 log.cc:824] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:43.462571 28562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2: No bootstrap required, opened a new log
I20260807 19:40:43.475276 28562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } }
I20260807 19:40:43.476006 28562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:43.476194 28562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d70c92c32c14225958a6819c6911de2, State: Initialized, Role: FOLLOWER
I20260807 19:40:43.477114 28562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } }
I20260807 19:40:43.479799 28408 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } attempt: 4
I20260807 19:40:43.484194 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.463333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54440 (local address 127.14.16.125:36345)
0807 19:40:43.463671 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:43.463676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:43.464730 (+ 1054us) server_negotiation.cc:407] Connection header received
0807 19:40:43.464948 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:43.464953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:43.465021 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:43.465086 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:43.468009 (+ 2923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.469231 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.471607 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.472254 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.472347 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.476417 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:43.476426 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:43.476433 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:43.476478 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:43.480394 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.481568 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.481576 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.481590 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.481745 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.482223 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.482228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.482231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.482654 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:43.482887 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:43.483570 (+ 683us) server_negotiation.cc:299] Negotiation successful
0807 19:40:43.483824 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223}
I20260807 19:40:43.485090 28566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.463340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59158 (local address 127.14.16.126:42905)
0807 19:40:43.463526 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:43.463532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:43.464491 (+ 959us) server_negotiation.cc:407] Connection header received
0807 19:40:43.464789 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:43.464796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:43.464891 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:43.464964 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:43.468317 (+ 3353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.469430 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.471770 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.472433 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.472529 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.476393 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:43.476399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:43.476404 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:43.476448 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:43.480879 (+ 4431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.482019 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.482028 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.482034 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.482245 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.482780 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.482789 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.482795 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.483252 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:43.483584 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:43.484271 (+ 687us) server_negotiation.cc:299] Negotiation successful
0807 19:40:43.484592 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:40:43.486344 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.464823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.14.16.124:46625)
0807 19:40:43.465019 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:43.465026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:43.467885 (+ 2859us) server_negotiation.cc:407] Connection header received
0807 19:40:43.468173 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:43.468178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:43.468249 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:43.468314 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:43.470694 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.471852 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.473870 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.474647 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.474761 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.479142 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:43.479150 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:43.479157 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:43.479199 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:43.482892 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.483826 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.483831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.483834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.484004 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.484467 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.484473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.484476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.484877 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:43.485131 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:43.485771 (+ 640us) server_negotiation.cc:299] Negotiation successful
0807 19:40:43.485993 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:40:43.492340 28577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } } }
I20260807 19:40:43.492841 28577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:40:43.494134 28562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 19:40:43 dist-test-slave-07c2 krb5kdc[28325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for kudu/127.14.16.124@KRBTEST.COM
I20260807 19:40:43.503269 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.481166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.14.16.124:46625)
0807 19:40:43.486827 (+ 5661us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:43.486832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:43.486849 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:40:43.486896 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:43.486901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:43.486972 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:43.487093 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:43.487998 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.488828 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.491000 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.491615 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.491757 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.495626 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:43.495636 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:43.495644 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:43.495687 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:43.500080 (+ 4393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.500905 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.500912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.500917 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.501081 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.501502 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.501510 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.501513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.501936 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:43.502083 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:43.502585 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 19:40:43.502844 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5565,"thread_start_us":10355,"threads_started":1}
I20260807 19:40:43.503507 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.480443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42746)
0807 19:40:43.480843 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.480863 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:43.480969 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:43.487123 (+ 6154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:43.487148 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:43.487655 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:43.487868 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:43.487882 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.488964 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.488972 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:43.490843 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:43.490863 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.492022 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.492028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:43.492252 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.493029 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:43.493050 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:43.495378 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:43.500229 (+ 4851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:43.500236 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:43.500242 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:43.500799 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:43.501197 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:43.501201 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:43.501205 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:43.501379 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:43.502100 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:43.502107 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:43.502303 (+ 196us) client_negotiation.cc:769] Sending connection context
0807 19:40:43.502465 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 19:40:43.502853 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252}
W20260807 19:40:43.507824 28589 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:40:43.508005 28589 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:40:43.512351 28408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38: Bootstrap starting.
I20260807 19:40:43.512960 28577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:40:43.513247 28577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } }
I20260807 19:40:43.516541 28408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:43.502565 28492 master_runner.cc:186] Error getting master registration for 127.14.16.126:42905: OK, instance_id { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" instance_seqno: 1786131642339991 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:40:43.517104 28492 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:40:43.517262 28577 leader_election.cc:288] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db42b8af284d49978d59abbf78ceaa38 (127.14.16.126:42905), 6cebf5b997f9446e99e539bcb514ba09 (127.14.16.125:36345)
I20260807 19:40:43.517962 28408 log.cc:824] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:43.521793 28408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38: No bootstrap required, opened a new log
I20260807 19:40:43.532811 28408 raft_consensus.cc:348] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } }
I20260807 19:40:43.533200 28408 raft_consensus.cc:374] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:43.533320 28408 raft_consensus.cc:729] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db42b8af284d49978d59abbf78ceaa38, State: Initialized, Role: FOLLOWER
I20260807 19:40:43.533958 28408 consensus_queue.cc:260] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } }
I20260807 19:40:43.535084 28592 sys_catalog.cc:455] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } } }
I20260807 19:40:43.535542 28592 sys_catalog.cc:458] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:40:43.536532 28408 sys_catalog.cc:564] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:40:43.536800 28382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d70c92c32c14225958a6819c6911de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db42b8af284d49978d59abbf78ceaa38" is_pre_election: true
I20260807 19:40:43.537428 28382 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d70c92c32c14225958a6819c6911de2 in term 0.
I20260807 19:40:43.539083 28508 leader_election.cc:302] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d70c92c32c14225958a6819c6911de2, db42b8af284d49978d59abbf78ceaa38; no voters:
I20260807 19:40:43.539641 28577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:40:43.539816 28577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:40:43.539940 28577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:43.543653 28577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } }
I20260807 19:40:43.544927 28382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d70c92c32c14225958a6819c6911de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db42b8af284d49978d59abbf78ceaa38"
I20260807 19:40:43.545596 28382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:43.548337 28577 leader_election.cc:288] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [CANDIDATE]: Term 1 election: Requested vote from peers db42b8af284d49978d59abbf78ceaa38 (127.14.16.126:42905), 6cebf5b997f9446e99e539bcb514ba09 (127.14.16.125:36345)
I20260807 19:40:43.549221 28382 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d70c92c32c14225958a6819c6911de2 in term 1.
I20260807 19:40:43.549724 28508 leader_election.cc:302] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d70c92c32c14225958a6819c6911de2, db42b8af284d49978d59abbf78ceaa38; no voters:
I20260807 19:40:43.550102 28577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:40:43.550884 28577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d70c92c32c14225958a6819c6911de2, State: Running, Role: LEADER
I20260807 19:40:43.551574 28577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } }
I20260807 19:40:43.562145 28601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d70c92c32c14225958a6819c6911de2. Latest consensus state: current_term: 1 leader_uuid: "3d70c92c32c14225958a6819c6911de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } } }
I20260807 19:40:43.562551 28601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:43.563500 28604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:40:43.565882 28604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:40:43.578879 28485 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } attempt: 5
I20260807 19:40:43.570382 28338 master_runner.cc:186] Error getting master registration for 127.14.16.125:36345: OK, instance_id { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" instance_seqno: 1786131642854559 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260807 19:40:43.579926 28360 tablet.cc:2369] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:40:43.583336 28338 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:40:43.578781 28382 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [term 1 FOLLOWER]: Refusing update from remote peer 3d70c92c32c14225958a6819c6911de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:40:43.586066 28577 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 07 19:40:43 dist-test-slave-07c2 krb5kdc[28325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131642, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.125@KRBTEST.COM for kudu/127.14.16.124@KRBTEST.COM
W20260807 19:40:43.594709 28606 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:40:43.594843 28606 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:40:43.606670 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.580251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.14.16.124:46625)
0807 19:40:43.580429 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:43.580435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:43.581179 (+ 744us) server_negotiation.cc:407] Connection header received
0807 19:40:43.581260 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:43.581265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:43.581342 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:43.581439 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:43.583023 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.584220 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.589734 (+ 5514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.590581 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.590736 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.596009 (+ 5273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:43.596017 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:43.596022 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:43.596064 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:43.600293 (+ 4229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.601694 (+ 1401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.601702 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.601706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.601923 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:43.602652 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:43.602658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:43.602661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:43.603403 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:43.603627 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:43.604804 (+ 1177us) server_negotiation.cc:299] Negotiation successful
0807 19:40:43.605259 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:40:43.607667 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:43.580640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46625 (local address 127.0.0.1:42760)
0807 19:40:43.580880 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:43.580898 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:43.581017 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:43.581693 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:43.581698 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:43.582276 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:43.582516 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:43.582530 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.587628 (+ 5098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.587638 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:43.589463 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:43.589486 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:43.590998 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:43.591004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:43.591582 (+ 578us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:43.592464 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:43.592488 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:43.595686 (+ 3198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:43.600569 (+ 4883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:43.600575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:43.600579 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:43.601489 (+ 910us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:43.602179 (+ 690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:43.602184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:43.602188 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:43.602439 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:43.603803 (+ 1364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:43.603808 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:43.604065 (+ 257us) client_negotiation.cc:769] Sending connection context
0807 19:40:43.605430 (+ 1365us) client_negotiation.cc:240] Negotiation successful
0807 19:40:43.606510 (+ 1080us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260807 19:40:43.609892 28592 sys_catalog.cc:455] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d70c92c32c14225958a6819c6911de2. Latest consensus state: current_term: 1 leader_uuid: "3d70c92c32c14225958a6819c6911de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } } }
I20260807 19:40:43.610224 28592 sys_catalog.cc:458] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:40:43.615023 28485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09: Bootstrap starting.
I20260807 19:40:43.616652 28592 sys_catalog.cc:455] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d70c92c32c14225958a6819c6911de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } } }
I20260807 19:40:43.616912 28592 sys_catalog.cc:458] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:40:43.617549 28607 mvcc.cc:205] Tried to move back new op lower bound from 7315995212080046080 to 7315995211996143616. Current Snapshot: MvccSnapshot[applied={T|T < 7315995212080046080}]
I20260807 19:40:43.626843 28485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:43.631114 28485 log.cc:824] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:43.632124 28609 mvcc.cc:205] Tried to move back new op lower bound from 7315995212080046080 to 7315995211996143616. Current Snapshot: MvccSnapshot[applied={T|T < 7315995212080046080}]
I20260807 19:40:43.637066 28577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d70c92c32c14225958a6819c6911de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } } }
I20260807 19:40:43.637370 28577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:43.638078 28604 catalog_manager.cc:1347] Generated new cluster ID: 3a6b826e34ab48cbae2f3b2c0c1649da
I20260807 19:40:43.638164 28604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:40:43.662962 28485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09: No bootstrap required, opened a new log
I20260807 19:40:43.692647 28604 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:40:43.696467 28604 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:40:43.696374 28485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } }
I20260807 19:40:43.697484 28485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:43.697791 28485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cebf5b997f9446e99e539bcb514ba09, State: Initialized, Role: FOLLOWER
I20260807 19:40:43.699131 28485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } }
I20260807 19:40:43.719014 28604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2: Generated new TSK 0
I20260807 19:40:43.718267 28614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } } }
I20260807 19:40:43.721473 28614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:40:43.722280 28604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:40:43.723538 28459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d70c92c32c14225958a6819c6911de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cebf5b997f9446e99e539bcb514ba09"
I20260807 19:40:43.723567 28460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d70c92c32c14225958a6819c6911de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cebf5b997f9446e99e539bcb514ba09" is_pre_election: true
I20260807 19:40:43.724115 28459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:43.727638 28459 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d70c92c32c14225958a6819c6911de2 in term 1.
I20260807 19:40:43.728470 28458 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [term 1 FOLLOWER]: Refusing update from remote peer 3d70c92c32c14225958a6819c6911de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:40:43.729187 28485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:40:43.729387 28601 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:40:43.795387 28620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d70c92c32c14225958a6819c6911de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } } }
I20260807 19:40:43.795902 28620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:40:43.795747 28614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d70c92c32c14225958a6819c6911de2. Latest consensus state: current_term: 1 leader_uuid: "3d70c92c32c14225958a6819c6911de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } } }
I20260807 19:40:43.796092 28614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:40:43.826882 28635 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:40:44.206573 28601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d70c92c32c14225958a6819c6911de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db42b8af284d49978d59abbf78ceaa38" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 42905 } } peers { permanent_uuid: "6cebf5b997f9446e99e539bcb514ba09" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 36345 } } peers { permanent_uuid: "3d70c92c32c14225958a6819c6911de2" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46625 } } }
I20260807 19:40:44.206911 28601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:44.597013 28606 catalog_manager.cc:1259] Loaded cluster ID: 3a6b826e34ab48cbae2f3b2c0c1649da
I20260807 19:40:44.597157 28606 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38: loading cluster ID for follower catalog manager: success
I20260807 19:40:44.599879 28606 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38: acquiring CA information for follower catalog manager: success
I20260807 19:40:44.601693 28606 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db42b8af284d49978d59abbf78ceaa38: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:40:44.831521 28635 catalog_manager.cc:1259] Loaded cluster ID: 3a6b826e34ab48cbae2f3b2c0c1649da
I20260807 19:40:44.831666 28635 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09: loading cluster ID for follower catalog manager: success
I20260807 19:40:44.834196 28635 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09: acquiring CA information for follower catalog manager: success
I20260807 19:40:44.836112 28635 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6cebf5b997f9446e99e539bcb514ba09: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:40:44.869019 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 28338
I20260807 19:40:44.870244 28402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:44.983497 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 28338
I20260807 19:40:45.017282 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 28415
I20260807 19:40:45.018519 28480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:45.039006 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:45.038149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:42905 (local address 127.0.0.1:59162)
0807 19:40:45.038629 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:45.038776 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:42905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":235,"threads_started":1}
W20260807 19:40:45.039390 28508 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.16.126:42905: connect: Connection refused (error 111)
I20260807 19:40:45.040923 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:45.040467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:42905 (local address 127.0.0.1:59176)
0807 19:40:45.040651 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:45.040758 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:42905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 19:40:45.041162 28508 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3d70c92c32c14225958a6819c6911de2 -> Peer db42b8af284d49978d59abbf78ceaa38 (127.14.16.126:42905): Couldn't send request to peer db42b8af284d49978d59abbf78ceaa38. Status: Network error: Client connection negotiation failed: client connection to 127.14.16.126:42905: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:40:45.129122 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 28415
I20260807 19:40:45.151496 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 28492
I20260807 19:40:45.152580 28557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:45.264338 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 28492
2026-08-07T19:40:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4820 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:40:45 dist-test-slave-07c2 krb5kdc[28645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:40:45 dist-test-slave-07c2 krb5kdc[28645](info): set up 2 sockets
Aug 07 19:40:45 dist-test-slave-07c2 krb5kdc[28645](info): commencing operation
krb5kdc: starting...
W20260807 19:40:46.411010 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.002s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:40:46 dist-test-slave-07c2 krb5kdc[28645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:40:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:40:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:40:46.690913 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:36381
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:35953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:36381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:40:47.045430 28658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:47.045845 28658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:47.046053 28658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:47.054095 28658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:40:47.054212 28658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:47.054261 28658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:47.054319 28658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:40:47.054366 28658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:40:47.064347 28658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:35953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:36381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:36381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:47.065685 28658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:47.067052 28658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:47.079929 28666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:47.080088 28663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:47.080106 28664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:47.080960 28658 server_base.cc:1034] running on GCE node
I20260807 19:40:47.081795 28658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:47.083321 28658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:47.084576 28658 hybrid_clock.cc:648] HybridClock initialized: now 1786131647084521 us; error 42 us; skew 500 ppm
I20260807 19:40:47.084969 28658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:47 dist-test-slave-07c2 krb5kdc[28645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131647, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:47.088286 28658 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:40:47.090014 28658 webserver.cc:460] Webserver started at http://127.14.16.126:44351/ using document root <none> and password file <none>
I20260807 19:40:47.090761 28658 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:47.090915 28658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:47.091234 28658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:40:47.094368 28658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "2cd3cb9e3a1345a8802376db8cf73351"
format_stamp: "Formatted at 2026-08-07 19:40:47 on dist-test-slave-07c2"
server_key: "12bf228e8c480acbc62cb4b7e40a031c"
server_key_iv: "0404f3b1c3d4fc9176304e223dd3b25f"
server_key_version: "encryptionkey@0"
I20260807 19:40:47.095173 28658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "2cd3cb9e3a1345a8802376db8cf73351"
format_stamp: "Formatted at 2026-08-07 19:40:47 on dist-test-slave-07c2"
server_key: "12bf228e8c480acbc62cb4b7e40a031c"
server_key_iv: "0404f3b1c3d4fc9176304e223dd3b25f"
server_key_version: "encryptionkey@0"
I20260807 19:40:47.100590 28658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:40:47.104283 28673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:47.105640 28658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 19:40:47.105840 28658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "2cd3cb9e3a1345a8802376db8cf73351"
format_stamp: "Formatted at 2026-08-07 19:40:47 on dist-test-slave-07c2"
server_key: "12bf228e8c480acbc62cb4b7e40a031c"
server_key_iv: "0404f3b1c3d4fc9176304e223dd3b25f"
server_key_version: "encryptionkey@0"
I20260807 19:40:47.106060 28658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:47.132186 28658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:47.136807 28658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:47.137131 28658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:47.158155 28658 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:36381
I20260807 19:40:47.158160 28727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:36381 every 8 connection(s)
I20260807 19:40:47.159806 28658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:40:47.163805 28728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:47.168916 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 28658
I20260807 19:40:47.169148 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:40:47.169813 14401 external_mini_cluster.cc:1428] Setting key 389508a4a66220e1ec069e9dce202936
I20260807 19:40:47.174932 28728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351: Bootstrap starting.
I20260807 19:40:47.178814 28728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:47.180154 28728 log.cc:824] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:47.183384 28728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351: No bootstrap required, opened a new log
Aug 07 19:40:47 dist-test-slave-07c2 krb5kdc[28645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:47.193094 28728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd3cb9e3a1345a8802376db8cf73351" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36381 } }
I20260807 19:40:47.193622 28728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:47.193771 28728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd3cb9e3a1345a8802376db8cf73351, State: Initialized, Role: FOLLOWER
I20260807 19:40:47.194389 28728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd3cb9e3a1345a8802376db8cf73351" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36381 } }
I20260807 19:40:47.194633 28728 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:40:47.194820 28728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:40:47.195016 28728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:47.195186 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:47.171738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.14.16.126:36381)
0807 19:40:47.172698 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:47.172712 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:47.172757 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:40:47.174466 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:47.174489 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:47.174880 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:47.175275 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:47.177300 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:47.178621 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:47.180753 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:47.181839 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:47.182035 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:47.186779 (+ 4744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:47.186787 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:47.186802 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:47.186841 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:47.190230 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:47.191254 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:47.191264 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:47.191271 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:47.191542 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:47.191974 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:47.191978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:47.191980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:47.192546 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:47.192781 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:47.193247 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 19:40:47.193885 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":481,"threads_started":1}
I20260807 19:40:47.197490 28728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd3cb9e3a1345a8802376db8cf73351" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36381 } }
I20260807 19:40:47.198092 28728 leader_election.cc:302] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cd3cb9e3a1345a8802376db8cf73351; no voters:
I20260807 19:40:47.198766 28728 leader_election.cc:288] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:40:47.198912 28733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:40:47.199281 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:40:47.200958 28728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:40:47.202425 28733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [term 1 LEADER]: Becoming Leader. State: Replica: 2cd3cb9e3a1345a8802376db8cf73351, State: Running, Role: LEADER
I20260807 19:40:47.203037 28733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd3cb9e3a1345a8802376db8cf73351" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36381 } }
I20260807 19:40:47.206280 28734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cd3cb9e3a1345a8802376db8cf73351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd3cb9e3a1345a8802376db8cf73351" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36381 } } }
I20260807 19:40:47.206746 28734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:47.207257 28735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cd3cb9e3a1345a8802376db8cf73351. Latest consensus state: current_term: 1 leader_uuid: "2cd3cb9e3a1345a8802376db8cf73351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd3cb9e3a1345a8802376db8cf73351" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36381 } } }
I20260807 19:40:47.207587 28735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:47.207808 28747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:40:47.212536 28747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:40:47.220898 28747 catalog_manager.cc:1347] Generated new cluster ID: 9e11be85f661480f9528cc75e409c474
I20260807 19:40:47.221025 28747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:40:47.237675 28747 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:40:47.238924 28747 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:40:47.246073 28747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cd3cb9e3a1345a8802376db8cf73351: Generated new TSK 0
I20260807 19:40:47.246869 28747 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:40:47.261732 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 28658
I20260807 19:40:47.263108 28723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:47.355338 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 28658
2026-08-07T19:40:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2086 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:40:47 dist-test-slave-07c2 krb5kdc[28757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:40:47 dist-test-slave-07c2 krb5kdc[28757](info): set up 2 sockets
Aug 07 19:40:47 dist-test-slave-07c2 krb5kdc[28757](info): commencing operation
krb5kdc: starting...
W20260807 19:40:48.494524 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.003s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:40:48 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:40:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:40:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:40:48.773277 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:45053
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:45053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:40:49.109539 28771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:49.109838 28771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:49.109930 28771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:49.117197 28771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:40:49.117314 28771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:49.117367 28771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:49.117439 28771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:40:49.117484 28771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:40:49.127141 28771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:45053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:45053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:49.128504 28771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:49.129781 28771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:49.141304 28779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:49.141340 28777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:49.141263 28776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:49.142405 28771 server_base.cc:1034] running on GCE node
I20260807 19:40:49.143225 28771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:49.144634 28771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:49.145895 28771 hybrid_clock.cc:648] HybridClock initialized: now 1786131649145847 us; error 51 us; skew 500 ppm
I20260807 19:40:49.146245 28771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:49 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131649, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:49.149370 28771 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:40:49.150924 28771 webserver.cc:460] Webserver started at http://127.14.16.126:36857/ using document root <none> and password file <none>
I20260807 19:40:49.151733 28771 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:49.151885 28771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:49.152201 28771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:40:49.155182 28771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "46a72cb93f2543c595d36469b7e2d500"
format_stamp: "Formatted at 2026-08-07 19:40:49 on dist-test-slave-07c2"
server_key: "6b0c161815344c79a18615565b48a31b"
server_key_iv: "0146d381f9316b621ee911ec649e2fe4"
server_key_version: "encryptionkey@0"
I20260807 19:40:49.156026 28771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "46a72cb93f2543c595d36469b7e2d500"
format_stamp: "Formatted at 2026-08-07 19:40:49 on dist-test-slave-07c2"
server_key: "6b0c161815344c79a18615565b48a31b"
server_key_iv: "0146d381f9316b621ee911ec649e2fe4"
server_key_version: "encryptionkey@0"
I20260807 19:40:49.161072 28771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:40:49.164461 28786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:49.165645 28771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:40:49.165843 28771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "46a72cb93f2543c595d36469b7e2d500"
format_stamp: "Formatted at 2026-08-07 19:40:49 on dist-test-slave-07c2"
server_key: "6b0c161815344c79a18615565b48a31b"
server_key_iv: "0146d381f9316b621ee911ec649e2fe4"
server_key_version: "encryptionkey@0"
I20260807 19:40:49.166036 28771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:49.177886 28771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:49.181748 28771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:49.182030 28771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:49.201610 28771 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:45053
I20260807 19:40:49.201653 28842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:45053 every 8 connection(s)
I20260807 19:40:49.203126 28771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:40:49.206980 28843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:49.210887 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 28771
I20260807 19:40:49.211135 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:40:49.212034 14401 external_mini_cluster.cc:1428] Setting key 41263c323f1e66538bac3f7c71628931
I20260807 19:40:49.221328 28843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Bootstrap starting.
I20260807 19:40:49.225369 28843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:49.226778 28843 log.cc:824] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:40:49 dist-test-slave-07c2 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:49.229952 28843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: No bootstrap required, opened a new log
I20260807 19:40:49.238298 28846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:49.214304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48066 (local address 127.14.16.126:45053)
0807 19:40:49.215185 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:49.215197 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:49.215241 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:40:49.216975 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:49.217000 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:49.217438 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:49.217796 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:49.219668 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:49.221010 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:49.222951 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:49.223743 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:49.223884 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:49.229275 (+ 5391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:49.229284 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:49.229299 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:49.229340 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:49.233388 (+ 4048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:49.234110 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:49.234118 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:49.234127 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:49.234359 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:49.234748 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:49.234753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:49.234758 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:49.235500 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:49.235880 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:49.236373 (+ 493us) server_negotiation.cc:299] Negotiation successful
0807 19:40:49.236872 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":430,"threads_started":1}
I20260807 19:40:49.240569 28843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } }
I20260807 19:40:49.240935 28843 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:49.241065 28843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a72cb93f2543c595d36469b7e2d500, State: Initialized, Role: FOLLOWER
I20260807 19:40:49.241663 28843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } }
I20260807 19:40:49.241858 28843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:40:49.241954 28843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:40:49.242080 28843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:49.244506 28843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } }
I20260807 19:40:49.244927 28843 leader_election.cc:302] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46a72cb93f2543c595d36469b7e2d500; no voters:
I20260807 19:40:49.245471 28843 leader_election.cc:288] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:40:49.245633 28848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:40:49.246444 28848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 1 LEADER]: Becoming Leader. State: Replica: 46a72cb93f2543c595d36469b7e2d500, State: Running, Role: LEADER
I20260807 19:40:49.246929 28848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } }
I20260807 19:40:49.247604 28843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:40:49.249943 28850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46a72cb93f2543c595d36469b7e2d500. Latest consensus state: current_term: 1 leader_uuid: "46a72cb93f2543c595d36469b7e2d500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } } }
I20260807 19:40:49.249933 28849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46a72cb93f2543c595d36469b7e2d500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } } }
I20260807 19:40:49.250437 28850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:49.250437 28849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:49.255513 28856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:40:49.260943 28856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:40:49.270386 28856 catalog_manager.cc:1347] Generated new cluster ID: fc3fceb082bc4e588cdd6312e2166472
I20260807 19:40:49.270517 28856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:40:49.287959 28856 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:40:49.289559 28856 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:40:49.298481 28856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Generated new TSK 0
I20260807 19:40:49.299412 28856 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:40:49.374663 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:45053
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:40:49.722777 28869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:49.723061 28869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:49.723227 28869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:49.730440 28869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:49.730552 28869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:49.730791 28869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:40:49.743698 28869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:45053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:49.745056 28869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:49.746359 28869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:49.759799 28874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:49.759845 28875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:49.759876 28878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:49.761006 28869 server_base.cc:1034] running on GCE node
I20260807 19:40:49.761777 28869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:49.762836 28869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:49.764173 28869 hybrid_clock.cc:648] HybridClock initialized: now 1786131649764129 us; error 50 us; skew 500 ppm
I20260807 19:40:49.764528 28869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:49 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131649, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:49.767761 28869 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:40:49.769272 28869 webserver.cc:460] Webserver started at http://127.14.16.65:44273/ using document root <none> and password file <none>
I20260807 19:40:49.770045 28869 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:49.770197 28869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:49.770524 28869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:40:49.773622 28869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "d186475b246e438a864cc5e6b6248aa0"
format_stamp: "Formatted at 2026-08-07 19:40:49 on dist-test-slave-07c2"
server_key: "b019ce05a9243a6bd803717ec1a6df80"
server_key_iv: "01239ac28b657747adec04df165c7a5d"
server_key_version: "encryptionkey@0"
I20260807 19:40:49.774348 28869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "d186475b246e438a864cc5e6b6248aa0"
format_stamp: "Formatted at 2026-08-07 19:40:49 on dist-test-slave-07c2"
server_key: "b019ce05a9243a6bd803717ec1a6df80"
server_key_iv: "01239ac28b657747adec04df165c7a5d"
server_key_version: "encryptionkey@0"
I20260807 19:40:49.779551 28869 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:40:49.783010 28885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:49.784493 28869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:40:49.784662 28869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "d186475b246e438a864cc5e6b6248aa0"
format_stamp: "Formatted at 2026-08-07 19:40:49 on dist-test-slave-07c2"
server_key: "b019ce05a9243a6bd803717ec1a6df80"
server_key_iv: "01239ac28b657747adec04df165c7a5d"
server_key_version: "encryptionkey@0"
I20260807 19:40:49.784839 28869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:49.802296 28869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:49.806489 28869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:49.806763 28869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:49.807857 28869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:49.809265 28869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:49.809340 28869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:49.809428 28869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:49.809473 28869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:49.845855 28869 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:44949
I20260807 19:40:49.845949 29001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:44949 every 8 connection(s)
I20260807 19:40:49.847371 28869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:40:49.853339 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 28869
I20260807 19:40:49.853569 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:40:49.854327 14401 external_mini_cluster.cc:1428] Setting key 9a33e42f830e1041f2295b54eb8cf5aa
Aug 07 19:40:49 dist-test-slave-07c2 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131649, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:49.871136 29004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:49.850150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:43797 (local address 127.14.16.126:45053)
0807 19:40:49.850622 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:49.850627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:49.851663 (+ 1036us) server_negotiation.cc:407] Connection header received
0807 19:40:49.853613 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:49.853621 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:49.853714 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:49.853856 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:49.855583 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:49.856451 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:49.858951 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:49.859401 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:49.859497 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:49.864031 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:49.864040 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:49.864048 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:49.864097 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:49.867739 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:49.868503 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:49.868511 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:49.868517 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:49.868688 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:49.869258 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:49.869262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:49.869264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:49.869615 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:49.869732 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:49.870698 (+ 966us) server_negotiation.cc:299] Negotiation successful
0807 19:40:49.870878 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":229,"threads_started":1}
I20260807 19:40:49.872892 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:49.850728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:43797)
0807 19:40:49.851507 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:49.851547 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:49.853261 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:49.854038 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:49.854046 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:49.854643 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:49.855404 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:49.855424 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:49.856661 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:49.856667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:49.858792 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:49.858811 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:49.859629 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:49.859640 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:49.859984 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:49.860768 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:49.860802 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:49.863775 (+ 2973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:49.867891 (+ 4116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:49.867898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:49.867911 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:49.868361 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:49.868827 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:49.868835 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:49.868840 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:49.869125 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:49.869746 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:49.869756 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:49.870234 (+ 478us) client_negotiation.cc:769] Sending connection context
0807 19:40:49.870582 (+ 348us) client_negotiation.cc:240] Negotiation successful
0807 19:40:49.871180 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":289,"threads_started":1}
I20260807 19:40:49.874611 29002 heartbeater.cc:347] Connected to a master server at 127.14.16.126:45053
I20260807 19:40:49.874928 29002 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:49.875761 29002 heartbeater.cc:511] Master 127.14.16.126:45053 requested a full tablet report, sending...
I20260807 19:40:49.877933 28807 ts_manager.cc:194] Registered new tserver with Master: d186475b246e438a864cc5e6b6248aa0 (127.14.16.65:44949)
I20260807 19:40:49.879483 28807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:43797
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:40:49.919183 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:45053
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:40:50.252408 29008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:50.252724 29008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:50.252889 29008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:50.260382 29008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:50.260553 29008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:50.260737 29008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:40:50.270959 29008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:45053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:50.272387 29008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:50.273737 29008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:50.287468 29014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:50.287559 29016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:50.287556 29013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:50.288443 29008 server_base.cc:1034] running on GCE node
I20260807 19:40:50.289162 29008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:50.290158 29008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:50.291436 29008 hybrid_clock.cc:648] HybridClock initialized: now 1786131650291392 us; error 54 us; skew 500 ppm
I20260807 19:40:50.291829 29008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:50 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131650, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:50.295223 29008 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:40:50.296895 29008 webserver.cc:460] Webserver started at http://127.14.16.66:43473/ using document root <none> and password file <none>
I20260807 19:40:50.297644 29008 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:50.297799 29008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:50.298115 29008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:40:50.301355 29008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "9d976c31385749c9b56afd3b415ae94d"
format_stamp: "Formatted at 2026-08-07 19:40:50 on dist-test-slave-07c2"
server_key: "640cbb75eb177fc5924ce2c1e1fab09e"
server_key_iv: "acc501084904d5184f208fac428f1ff9"
server_key_version: "encryptionkey@0"
I20260807 19:40:50.302191 29008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "9d976c31385749c9b56afd3b415ae94d"
format_stamp: "Formatted at 2026-08-07 19:40:50 on dist-test-slave-07c2"
server_key: "640cbb75eb177fc5924ce2c1e1fab09e"
server_key_iv: "acc501084904d5184f208fac428f1ff9"
server_key_version: "encryptionkey@0"
I20260807 19:40:50.307617 29008 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:40:50.311093 29023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:50.312474 29008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:40:50.312650 29008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "9d976c31385749c9b56afd3b415ae94d"
format_stamp: "Formatted at 2026-08-07 19:40:50 on dist-test-slave-07c2"
server_key: "640cbb75eb177fc5924ce2c1e1fab09e"
server_key_iv: "acc501084904d5184f208fac428f1ff9"
server_key_version: "encryptionkey@0"
I20260807 19:40:50.312872 29008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:50.324954 29008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:50.329310 29008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:50.329614 29008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:50.330564 29008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:50.331944 29008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:50.332047 29008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:50.332197 29008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:50.332271 29008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:50.367136 29008 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:33275
I20260807 19:40:50.367182 29141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:33275 every 8 connection(s)
I20260807 19:40:50.368458 29008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:40:50.376080 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 29008
I20260807 19:40:50.376303 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:40:50.376808 14401 external_mini_cluster.cc:1428] Setting key 4e26915fc13d55efb866c8ebcbd09ab4
Aug 07 19:40:50 dist-test-slave-07c2 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131650, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:50.389644 29004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:50.371108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:42555 (local address 127.14.16.126:45053)
0807 19:40:50.371360 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:50.371367 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:50.372554 (+ 1187us) server_negotiation.cc:407] Connection header received
0807 19:40:50.374471 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:50.374475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:50.374542 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:50.374630 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:50.376400 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:50.377186 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:50.379102 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:50.379578 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:50.379678 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:50.383848 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:50.383852 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:50.383855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:50.383886 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:50.386526 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:50.387253 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:50.387257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:50.387260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:50.387441 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:50.387877 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:50.387882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:50.387886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:50.388192 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:50.388351 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:50.389049 (+ 698us) server_negotiation.cc:299] Negotiation successful
0807 19:40:50.389291 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:40:50.390646 29144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:50.371708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:42555)
0807 19:40:50.372400 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:50.372437 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:50.374227 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:50.374840 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:50.374847 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:50.375429 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:50.376199 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:50.376226 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:50.377349 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:50.377357 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:50.378954 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:50.378976 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:50.379816 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:50.379826 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:50.380141 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:50.380879 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:50.380914 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:50.383622 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:50.386668 (+ 3046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:50.386678 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:50.386695 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:50.387115 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:50.387554 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:50.387559 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:50.387562 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:50.387755 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:50.388324 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:50.388332 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:50.388672 (+ 340us) client_negotiation.cc:769] Sending connection context
0807 19:40:50.388950 (+ 278us) client_negotiation.cc:240] Negotiation successful
0807 19:40:50.389521 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":278,"threads_started":1}
I20260807 19:40:50.392179 29142 heartbeater.cc:347] Connected to a master server at 127.14.16.126:45053
I20260807 19:40:50.392493 29142 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:50.393183 29142 heartbeater.cc:511] Master 127.14.16.126:45053 requested a full tablet report, sending...
I20260807 19:40:50.394696 28807 ts_manager.cc:194] Registered new tserver with Master: 9d976c31385749c9b56afd3b415ae94d (127.14.16.66:33275)
I20260807 19:40:50.395987 28807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:42555
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:40:50.442374 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:45053
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:40:50.801570 29147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:50.801857 29147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:50.801954 29147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:50.809346 29147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:50.809453 29147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:50.809567 29147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:40:50.819603 29147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:45053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:50.820847 29147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:50.822062 29147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:50.836030 29153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:50.836125 29152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:50.836165 29155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:50.837036 29147 server_base.cc:1034] running on GCE node
I20260807 19:40:50.837592 29147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:50.838518 29147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:50.839757 29147 hybrid_clock.cc:648] HybridClock initialized: now 1786131650839683 us; error 30 us; skew 500 ppm
I20260807 19:40:50.840087 29147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:50 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131650, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:50.843195 29147 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:40:50.844678 29147 webserver.cc:460] Webserver started at http://127.14.16.67:44351/ using document root <none> and password file <none>
I20260807 19:40:50.845575 29147 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:50.845711 29147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:50.846087 29147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:40:50.850099 29147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "1fe054d2c0b146098eacaf66274d2559"
format_stamp: "Formatted at 2026-08-07 19:40:50 on dist-test-slave-07c2"
server_key: "3b80a2f3f661c3af206982828c4b5f6b"
server_key_iv: "39648077cab60751db805b30f4a6cb37"
server_key_version: "encryptionkey@0"
I20260807 19:40:50.851102 29147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "1fe054d2c0b146098eacaf66274d2559"
format_stamp: "Formatted at 2026-08-07 19:40:50 on dist-test-slave-07c2"
server_key: "3b80a2f3f661c3af206982828c4b5f6b"
server_key_iv: "39648077cab60751db805b30f4a6cb37"
server_key_version: "encryptionkey@0"
I20260807 19:40:50.856665 29147 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:40:50.860159 29162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:50.861457 29147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:40:50.861639 29147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "1fe054d2c0b146098eacaf66274d2559"
format_stamp: "Formatted at 2026-08-07 19:40:50 on dist-test-slave-07c2"
server_key: "3b80a2f3f661c3af206982828c4b5f6b"
server_key_iv: "39648077cab60751db805b30f4a6cb37"
server_key_version: "encryptionkey@0"
I20260807 19:40:50.861860 29147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:50.882584 29002 heartbeater.cc:503] Master 127.14.16.126:45053 was elected leader, sending a full tablet report...
I20260807 19:40:50.886708 29147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:50.890815 29147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:50.891099 29147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:50.892112 29147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:50.893448 29147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:50.893553 29147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:50.893699 29147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:50.893781 29147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:50.928062 29147 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:44471
I20260807 19:40:50.928110 29277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:44471 every 8 connection(s)
I20260807 19:40:50.929375 29147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:40:50.931510 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 29147
I20260807 19:40:50.931715 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:40:50.932308 14401 external_mini_cluster.cc:1428] Setting key 11aa88d9dc4be9850a43a8a8a6617541
Aug 07 19:40:50 dist-test-slave-07c2 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131650, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:50.950059 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:50.932064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:39907 (local address 127.14.16.126:45053)
0807 19:40:50.932563 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:50.932567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:50.935360 (+ 2793us) server_negotiation.cc:407] Connection header received
0807 19:40:50.936046 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:50.936050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:50.936113 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:50.936200 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:50.938073 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:50.938842 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:50.940401 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:50.940815 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:50.940909 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:50.944184 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:50.944189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:50.944192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:50.944225 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:50.947049 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:50.947998 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:50.948002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:50.948005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:50.948168 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:50.948593 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:50.948596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:50.948598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:50.948862 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:50.948980 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:50.949629 (+ 649us) server_negotiation.cc:299] Negotiation successful
0807 19:40:50.949802 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":290,"threads_started":1}
I20260807 19:40:50.950984 29281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:50.932670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:39907)
0807 19:40:50.933553 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:50.933602 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:50.935819 (+ 2217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:50.936481 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:50.936495 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:50.937329 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:50.937923 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:50.937940 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:50.938971 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:50.938974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:50.940266 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:50.940278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:50.940997 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:50.941002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:50.941197 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:50.941805 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:50.941826 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:50.943973 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:50.947225 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:50.947235 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:50.947252 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:50.947879 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:50.948286 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:50.948290 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:50.948294 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:50.948482 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:50.948982 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:50.948987 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:50.949292 (+ 305us) client_negotiation.cc:769] Sending connection context
0807 19:40:50.949522 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 19:40:50.949881 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":357,"threads_started":1}
I20260807 19:40:50.952272 29278 heartbeater.cc:347] Connected to a master server at 127.14.16.126:45053
I20260807 19:40:50.952576 29278 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:50.953282 29278 heartbeater.cc:511] Master 127.14.16.126:45053 requested a full tablet report, sending...
I20260807 19:40:50.954631 28807 ts_manager.cc:194] Registered new tserver with Master: 1fe054d2c0b146098eacaf66274d2559 (127.14.16.67:44471)
I20260807 19:40:50.955593 28807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:39907
I20260807 19:40:50.960264 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:40:50.977352 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:50.964861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.14.16.126:45053)
0807 19:40:50.965041 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:50.965050 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:50.965538 (+ 488us) server_negotiation.cc:407] Connection header received
0807 19:40:50.965736 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:50.965740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:50.965816 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:50.965913 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:50.967453 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:50.968182 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:50.969527 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:50.969924 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:50.970018 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:50.972544 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:50.972548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:50.972552 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:50.972581 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:50.974926 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:50.975717 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:50.975741 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:50.975745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:50.975868 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:50.976201 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:50.976204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:50.976206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:50.976513 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:50.976694 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:50.976969 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 19:40:50.977134 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 19:40:50.981227 28807 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:40:50.988086 28807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:40:51.011734 29297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.002947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47524 (local address 127.14.16.67:44471)
0807 19:40:51.003962 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:51.003968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:51.003993 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:40:51.004065 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:51.004070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:51.004303 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:51.004501 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:51.005310 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.006877 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.009554 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.010503 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.010667 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.010909 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:51.011157 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 19:40:51.011361 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":692,"threads_started":1}
I20260807 19:40:51.011807 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.002276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:44471 (local address 127.0.0.1:47524)
0807 19:40:51.002775 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:51.002792 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:51.002896 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:51.004511 (+ 1615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:51.004517 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:51.004548 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:51.005088 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:51.005106 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.007021 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.007027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:51.009390 (+ 2363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:51.009403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.010695 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.010701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:51.010895 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.010952 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 19:40:51.011110 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 19:40:51.011320 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":166,"thread_start_us":707,"threads_started":1}
I20260807 19:40:51.012352 29293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.000966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:44949 (local address 127.0.0.1:54024)
0807 19:40:51.002143 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:51.002162 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:51.002271 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:51.004178 (+ 1907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:51.004183 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:51.004208 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:51.004721 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:51.004736 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.006355 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.006362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:51.009487 (+ 3125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:51.009498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.011483 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.011489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:51.011695 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.011739 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 19:40:51.011835 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 19:40:51.011962 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1045,"mutex_wait_us":531,"thread_start_us":898,"threads_started":1}
I20260807 19:40:51.013195 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.000567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:33275 (local address 127.0.0.1:35056)
0807 19:40:51.001337 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:51.001385 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:51.001552 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:51.003507 (+ 1955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:51.003516 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:51.003546 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:51.003958 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:51.003970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.006791 (+ 2821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.006796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:51.009440 (+ 2644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:51.009465 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.011451 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.011462 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:51.011665 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.012566 (+ 901us) client_negotiation.cc:769] Sending connection context
0807 19:40:51.012663 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 19:40:51.012798 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":1057,"thread_start_us":386,"threads_started":1}
I20260807 19:40:51.013208 29294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.001589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35056 (local address 127.14.16.66:33275)
0807 19:40:51.002790 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:51.002800 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:51.002829 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:40:51.002929 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:51.002940 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:51.003224 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:51.004163 (+ 939us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:51.004886 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.006648 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.009655 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.011139 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.011742 (+ 603us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.011942 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:51.012704 (+ 762us) server_negotiation.cc:299] Negotiation successful
0807 19:40:51.012869 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":891,"threads_started":1}
I20260807 19:40:51.014220 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.002130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54024 (local address 127.14.16.65:44949)
0807 19:40:51.003611 (+ 1481us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:51.003618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:51.003647 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:40:51.003740 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:51.003749 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:51.003968 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:51.004163 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:51.005029 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.006203 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.009655 (+ 3452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.011136 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.013405 (+ 2269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.013640 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:51.013712 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 19:40:51.013886 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1302,"thread_start_us":1124,"threads_started":1}
I20260807 19:40:51.016374 29210 tablet_service.cc:1467] Processing CreateTablet for tablet 916bd1bafc9e48d58225633cee41b0e0 (DEFAULT_TABLE table=test-table [id=a290394f556049a3958867bfbd143f81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:40:51.017596 28934 tablet_service.cc:1467] Processing CreateTablet for tablet 916bd1bafc9e48d58225633cee41b0e0 (DEFAULT_TABLE table=test-table [id=a290394f556049a3958867bfbd143f81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:40:51.018756 28934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 916bd1bafc9e48d58225633cee41b0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:51.018981 29210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 916bd1bafc9e48d58225633cee41b0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:51.018780 29073 tablet_service.cc:1467] Processing CreateTablet for tablet 916bd1bafc9e48d58225633cee41b0e0 (DEFAULT_TABLE table=test-table [id=a290394f556049a3958867bfbd143f81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:40:51.020110 29073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 916bd1bafc9e48d58225633cee41b0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:51.029287 29298 tablet_bootstrap.cc:492] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0: Bootstrap starting.
I20260807 19:40:51.033176 29298 tablet_bootstrap.cc:654] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:51.034423 29299 tablet_bootstrap.cc:492] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d: Bootstrap starting.
I20260807 19:40:51.034427 29298 log.cc:824] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:51.038094 29298 tablet_bootstrap.cc:492] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0: No bootstrap required, opened a new log
I20260807 19:40:51.038120 29299 tablet_bootstrap.cc:654] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:51.038378 29298 ts_tablet_manager.cc:1397] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
I20260807 19:40:51.039433 29299 log.cc:824] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:51.043406 29300 tablet_bootstrap.cc:492] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559: Bootstrap starting.
I20260807 19:40:51.046279 29299 tablet_bootstrap.cc:492] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d: No bootstrap required, opened a new log
I20260807 19:40:51.046554 29299 ts_tablet_manager.cc:1397] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260807 19:40:51.047055 29300 tablet_bootstrap.cc:654] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:51.048190 29300 log.cc:824] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:51.050061 29298 raft_consensus.cc:348] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:51.050642 29298 raft_consensus.cc:374] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:51.050791 29298 raft_consensus.cc:729] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d186475b246e438a864cc5e6b6248aa0, State: Initialized, Role: FOLLOWER
I20260807 19:40:51.051215 29300 tablet_bootstrap.cc:492] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559: No bootstrap required, opened a new log
I20260807 19:40:51.051519 29300 ts_tablet_manager.cc:1397] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260807 19:40:51.051486 29298 consensus_queue.cc:260] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:51.053231 29298 ts_tablet_manager.cc:1428] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0: Time spent starting tablet: real 0.015s user 0.013s sys 0.003s
I20260807 19:40:51.056811 29299 raft_consensus.cc:348] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:51.057255 29299 raft_consensus.cc:374] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:51.057374 29299 raft_consensus.cc:729] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d976c31385749c9b56afd3b415ae94d, State: Initialized, Role: FOLLOWER
I20260807 19:40:51.057979 29299 consensus_queue.cc:260] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:51.059122 29142 heartbeater.cc:503] Master 127.14.16.126:45053 was elected leader, sending a full tablet report...
I20260807 19:40:51.059734 29299 ts_tablet_manager.cc:1428] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260807 19:40:51.060976 29300 raft_consensus.cc:348] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:51.061344 29300 raft_consensus.cc:374] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:51.061450 29300 raft_consensus.cc:729] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe054d2c0b146098eacaf66274d2559, State: Initialized, Role: FOLLOWER
I20260807 19:40:51.061996 29300 consensus_queue.cc:260] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:51.063109 29278 heartbeater.cc:503] Master 127.14.16.126:45053 was elected leader, sending a full tablet report...
I20260807 19:40:51.064043 29300 ts_tablet_manager.cc:1428] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260807 19:40:51.071749 29304 raft_consensus.cc:482] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:40:51.072003 29304 raft_consensus.cc:504] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:51.073580 29304 leader_election.cc:288] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d976c31385749c9b56afd3b415ae94d (127.14.16.66:33275), 1fe054d2c0b146098eacaf66274d2559 (127.14.16.67:44471)
I20260807 19:40:51.080644 29294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.073811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:35037 (local address 127.14.16.66:33275)
0807 19:40:51.074050 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:51.074059 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:51.074965 (+ 906us) server_negotiation.cc:407] Connection header received
0807 19:40:51.075186 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:51.075192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:51.075259 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:51.075381 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:51.076019 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.076742 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.078494 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.079580 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.079764 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.079836 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:51.080243 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 19:40:51.080371 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:40:51.080736 29297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.073979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:52299 (local address 127.14.16.67:44471)
0807 19:40:51.074161 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:51.074167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:51.075152 (+ 985us) server_negotiation.cc:407] Connection header received
0807 19:40:51.075378 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:51.075382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:51.075435 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:51.075529 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:51.076224 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.076926 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.078664 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.079781 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.079935 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.080007 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:51.080346 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 19:40:51.080474 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:40:51.080822 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.073766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:33275 (local address 127.14.16.65:35037)
0807 19:40:51.074849 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:51.074865 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:51.075046 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:51.075395 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:51.075400 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:51.075423 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:51.075874 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:51.075884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.076874 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.076880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:51.078336 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:51.078348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.079823 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.079832 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:51.080032 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.080105 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 19:40:51.080217 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 19:40:51.080347 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":20,"spinlock_wait_cycles":24960,"thread_start_us":717,"threads_started":1}
I20260807 19:40:51.081197 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.074017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:44471 (local address 127.14.16.65:52299)
0807 19:40:51.075053 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:51.075068 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:51.075212 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:51.075549 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:51.075554 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:51.075604 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:51.076058 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:51.076068 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.077070 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.077076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:51.078508 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:51.078519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.080026 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.080033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:51.080186 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.080209 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:40:51.080319 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 19:40:51.080440 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"mutex_wait_us":50,"spinlock_wait_cycles":768,"thread_start_us":717,"threads_started":1}
I20260807 19:40:51.081964 29093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916bd1bafc9e48d58225633cee41b0e0" candidate_uuid: "d186475b246e438a864cc5e6b6248aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d976c31385749c9b56afd3b415ae94d" is_pre_election: true
I20260807 19:40:51.081992 29230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916bd1bafc9e48d58225633cee41b0e0" candidate_uuid: "d186475b246e438a864cc5e6b6248aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe054d2c0b146098eacaf66274d2559" is_pre_election: true
I20260807 19:40:51.082461 29093 raft_consensus.cc:2454] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d186475b246e438a864cc5e6b6248aa0 in term 0.
I20260807 19:40:51.082512 29230 raft_consensus.cc:2454] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d186475b246e438a864cc5e6b6248aa0 in term 0.
I20260807 19:40:51.083196 28887 leader_election.cc:302] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe054d2c0b146098eacaf66274d2559, d186475b246e438a864cc5e6b6248aa0; no voters:
I20260807 19:40:51.083662 29304 raft_consensus.cc:2781] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:40:51.083837 29304 raft_consensus.cc:482] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:40:51.083945 29304 raft_consensus.cc:3037] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:51.086099 29304 raft_consensus.cc:504] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:51.086707 29304 leader_election.cc:288] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [CANDIDATE]: Term 1 election: Requested vote from peers 9d976c31385749c9b56afd3b415ae94d (127.14.16.66:33275), 1fe054d2c0b146098eacaf66274d2559 (127.14.16.67:44471)
I20260807 19:40:51.087132 29093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916bd1bafc9e48d58225633cee41b0e0" candidate_uuid: "d186475b246e438a864cc5e6b6248aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d976c31385749c9b56afd3b415ae94d"
I20260807 19:40:51.087244 29230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916bd1bafc9e48d58225633cee41b0e0" candidate_uuid: "d186475b246e438a864cc5e6b6248aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe054d2c0b146098eacaf66274d2559"
I20260807 19:40:51.087386 29093 raft_consensus.cc:3037] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:51.087472 29230 raft_consensus.cc:3037] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:51.090762 29230 raft_consensus.cc:2454] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d186475b246e438a864cc5e6b6248aa0 in term 1.
I20260807 19:40:51.090762 29093 raft_consensus.cc:2454] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d186475b246e438a864cc5e6b6248aa0 in term 1.
I20260807 19:40:51.091217 28887 leader_election.cc:302] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe054d2c0b146098eacaf66274d2559, d186475b246e438a864cc5e6b6248aa0; no voters:
I20260807 19:40:51.091532 29304 raft_consensus.cc:2781] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:40:51.092031 29304 raft_consensus.cc:686] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 1 LEADER]: Becoming Leader. State: Replica: d186475b246e438a864cc5e6b6248aa0, State: Running, Role: LEADER
I20260807 19:40:51.092504 29304 consensus_queue.cc:237] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:51.097229 28806 catalog_manager.cc:5495] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d186475b246e438a864cc5e6b6248aa0 (127.14.16.65). New cstate: current_term: 1 leader_uuid: "d186475b246e438a864cc5e6b6248aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } health_report { overall_health: UNKNOWN } } }
W20260807 19:40:51.099799 29003 tablet.cc:2369] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:40:51.120577 29143 tablet.cc:2369] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:40:51.135439 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.130308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54034 (local address 127.14.16.65:44949)
0807 19:40:51.130526 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:51.130530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:51.130548 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:40:51.130710 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:51.130714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:51.130797 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:51.130925 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:40:51.131341 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.132039 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.133395 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.133792 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.133886 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.134335 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:40:51.134819 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:40:51.134928 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:51.135032 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 19:40:51.135097 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:40:51.143558 29093 raft_consensus.cc:1261] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d [term 1 FOLLOWER]: Refusing update from remote peer d186475b246e438a864cc5e6b6248aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:40:51.143596 29230 raft_consensus.cc:1261] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559 [term 1 FOLLOWER]: Refusing update from remote peer d186475b246e438a864cc5e6b6248aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:40:51.144554 29309 consensus_queue.cc:1035] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:40:51.144918 29304 consensus_queue.cc:1035] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:40:51.157094 29312 mvcc.cc:205] Tried to move back new op lower bound from 7315995243073601536 to 7315995242878402560. Current Snapshot: MvccSnapshot[applied={T|T < 7315995243073601536}]
I20260807 19:40:51.159590 29314 mvcc.cc:205] Tried to move back new op lower bound from 7315995243073601536 to 7315995242878402560. Current Snapshot: MvccSnapshot[applied={T|T < 7315995243073601536 or (T in {7315995243073601536})}]
I20260807 19:40:51.172887 28806 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:40:51.173243 28806 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:40:51.177215 28806 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Sending DeleteTablet for 3 replicas of tablet 916bd1bafc9e48d58225633cee41b0e0
I20260807 19:40:51.178563 28934 tablet_service.cc:1514] Processing DeleteTablet for tablet 916bd1bafc9e48d58225633cee41b0e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:40:51 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:54024
I20260807 19:40:51.178550 29073 tablet_service.cc:1514] Processing DeleteTablet for tablet 916bd1bafc9e48d58225633cee41b0e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:40:51 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:35056
I20260807 19:40:51.178866 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 28869
I20260807 19:40:51.179611 29210 tablet_service.cc:1514] Processing DeleteTablet for tablet 916bd1bafc9e48d58225633cee41b0e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:40:51 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:47524
I20260807 19:40:51.220085 29322 tablet_replica.cc:330] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d: stopping tablet replica
I20260807 19:40:51.220757 29322 raft_consensus.cc:2229] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:40:51.221462 29322 raft_consensus.cc:2258] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:40:51.225333 29323 tablet_replica.cc:330] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0: stopping tablet replica
I20260807 19:40:51.225986 29325 tablet_replica.cc:330] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559: stopping tablet replica
I20260807 19:40:51.226579 29325 raft_consensus.cc:2229] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:40:51.227087 29323 raft_consensus.cc:2229] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:40:51.227692 29325 raft_consensus.cc:2258] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:40:51.229657 29322 ts_tablet_manager.cc:1905] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:40:51.229943 29323 raft_consensus.cc:2258] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:40:51.232378 29325 ts_tablet_manager.cc:1905] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:40:51.236693 29322 ts_tablet_manager.cc:1918] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:40:51.236877 29322 log.cc:1196] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/916bd1bafc9e48d58225633cee41b0e0
I20260807 19:40:51.237417 29322 ts_tablet_manager.cc:1939] T 916bd1bafc9e48d58225633cee41b0e0 P 9d976c31385749c9b56afd3b415ae94d: Deleting consensus metadata
I20260807 19:40:51.239199 28789 catalog_manager.cc:4841] TS 9d976c31385749c9b56afd3b415ae94d (127.14.16.66:33275): tablet 916bd1bafc9e48d58225633cee41b0e0 (table test-table [id=a290394f556049a3958867bfbd143f81]) successfully deleted
I20260807 19:40:51.241112 29323 ts_tablet_manager.cc:1905] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 19:40:51 dist-test-slave-07c2 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:40:51.243496 29325 ts_tablet_manager.cc:1918] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:40:51.243670 29325 log.cc:1196] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/916bd1bafc9e48d58225633cee41b0e0
I20260807 19:40:51.244234 29325 ts_tablet_manager.cc:1939] T 916bd1bafc9e48d58225633cee41b0e0 P 1fe054d2c0b146098eacaf66274d2559: Deleting consensus metadata
I20260807 19:40:51.246120 28788 catalog_manager.cc:4841] TS 1fe054d2c0b146098eacaf66274d2559 (127.14.16.67:44471): tablet 916bd1bafc9e48d58225633cee41b0e0 (table test-table [id=a290394f556049a3958867bfbd143f81]) successfully deleted
I20260807 19:40:51.248695 29323 ts_tablet_manager.cc:1918] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:40:51.248886 29323 log.cc:1196] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/916bd1bafc9e48d58225633cee41b0e0
I20260807 19:40:51.249442 29323 ts_tablet_manager.cc:1939] T 916bd1bafc9e48d58225633cee41b0e0 P d186475b246e438a864cc5e6b6248aa0: Deleting consensus metadata
I20260807 19:40:51.251197 28788 catalog_manager.cc:4841] TS d186475b246e438a864cc5e6b6248aa0 (127.14.16.65:44949): tablet 916bd1bafc9e48d58225633cee41b0e0 (table test-table [id=a290394f556049a3958867bfbd143f81]) successfully deleted
I20260807 19:40:51.251762 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.181071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.14.16.65:44949)
0807 19:40:51.181260 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:51.181266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:51.213745 (+ 32479us) server_negotiation.cc:407] Connection header received
0807 19:40:51.214143 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:51.214153 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:51.214311 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:51.214409 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:51.222082 (+ 7673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.223776 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.227607 (+ 3831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.228624 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.228760 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.244314 (+ 15554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:51.244328 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:51.244348 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:51.244404 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:51.248428 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:51.249428 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:51.249435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:51.249437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:51.249642 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:51.250051 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:51.250056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:51.250059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:51.250429 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:51.250792 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:51.251088 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 19:40:51.251352 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:40:51.252462 28997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:51.356135 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 28869
I20260807 19:40:51.381925 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 29008
Aug 07 19:40:51 dist-test-slave-07c2 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:40:51.397902 29294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.382757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35066 (local address 127.14.16.66:33275)
0807 19:40:51.382926 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:51.382932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:51.383041 (+ 109us) server_negotiation.cc:407] Connection header received
0807 19:40:51.383255 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:51.383266 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:51.383412 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:51.383532 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:51.384836 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.385618 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.386942 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.387582 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.387719 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.392049 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:51.392061 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:51.392075 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:51.392113 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:51.394994 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:51.395757 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:51.395765 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:51.395768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:51.395889 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:51.396295 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:51.396306 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:51.396309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:51.396682 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:51.397007 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:51.397346 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 19:40:51.397571 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:40:51.398538 29136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:51.490092 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 29008
I20260807 19:40:51.509394 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 29147
Aug 07 19:40:51 dist-test-slave-07c2 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:40:51.524621 29297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:51.510247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.14.16.67:44471)
0807 19:40:51.510429 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:51.510433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:51.510452 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:40:51.510576 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:51.510580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:51.510649 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:51.510757 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:51.512157 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.512946 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.514421 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:51.514833 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:51.514930 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:51.519210 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:51.519224 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:51.519240 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:51.519279 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:51.521951 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:51.522588 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:51.522596 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:51.522598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:51.522755 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:51.523136 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:51.523141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:51.523143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:51.523580 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:51.523825 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:51.524077 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 19:40:51.524243 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:40:51.525281 29273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:51.622507 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 29147
I20260807 19:40:51.641660 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 28771
I20260807 19:40:51.642925 28837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:51.753615 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 28771
I20260807 19:40:51.778064 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:45053
--webserver_interface=127.14.16.126
--webserver_port=36857
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:45053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:40:52.118970 29331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:52.119274 29331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:52.119494 29331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:52.126919 29331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:40:52.127027 29331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:52.127128 29331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:52.127233 29331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:40:52.127336 29331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:40:52.137609 29331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:45053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:45053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=36857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:52.138947 29331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:52.140291 29331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:52.167385 29338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:52.167472 29340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:52.167410 29337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:52.168576 29331 server_base.cc:1034] running on GCE node
I20260807 19:40:52.169593 29331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:52.171381 29331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:52.172654 29331 hybrid_clock.cc:648] HybridClock initialized: now 1786131652172607 us; error 59 us; skew 500 ppm
I20260807 19:40:52.173136 29331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:52 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131652, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:52.177356 29331 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:40:52.179366 29331 webserver.cc:460] Webserver started at http://127.14.16.126:36857/ using document root <none> and password file <none>
I20260807 19:40:52.180204 29331 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:52.180383 29331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:52.186697 29331 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 19:40:52.189971 29347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:52.191413 29331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:40:52.191609 29331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "46a72cb93f2543c595d36469b7e2d500"
format_stamp: "Formatted at 2026-08-07 19:40:49 on dist-test-slave-07c2"
server_key: "6b0c161815344c79a18615565b48a31b"
server_key_iv: "0146d381f9316b621ee911ec649e2fe4"
server_key_version: "encryptionkey@0"
I20260807 19:40:52.192775 29331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:52.207844 29331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:52.212872 29331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:52.213201 29331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:52.234967 29331 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:45053
I20260807 19:40:52.234989 29399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:45053 every 8 connection(s)
I20260807 19:40:52.236734 29331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:40:52.244064 29400 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:40:52.246851 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 29331
I20260807 19:40:52.247946 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:44949
--local_ip_for_outbound_sockets=127.14.16.65
--tserver_master_addrs=127.14.16.126:45053
--webserver_port=44273
--webserver_interface=127.14.16.65
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:40:52.248160 29400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Bootstrap starting.
I20260807 19:40:52.273391 29400 log.cc:824] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:52.286801 29400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:40:52.287483 29400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Bootstrap complete.
I20260807 19:40:52.297549 29400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } }
I20260807 19:40:52.298472 29400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a72cb93f2543c595d36469b7e2d500, State: Initialized, Role: FOLLOWER
I20260807 19:40:52.299141 29400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } }
I20260807 19:40:52.299433 29400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:40:52.299593 29400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:40:52.299789 29400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:40:52.303936 29400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } }
I20260807 19:40:52.304481 29400 leader_election.cc:302] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46a72cb93f2543c595d36469b7e2d500; no voters:
I20260807 19:40:52.305155 29400 leader_election.cc:288] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:40:52.305312 29404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:40:52.306782 29404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 2 LEADER]: Becoming Leader. State: Replica: 46a72cb93f2543c595d36469b7e2d500, State: Running, Role: LEADER
I20260807 19:40:52.307458 29404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } }
I20260807 19:40:52.307688 29400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:40:52.310294 29406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46a72cb93f2543c595d36469b7e2d500. Latest consensus state: current_term: 2 leader_uuid: "46a72cb93f2543c595d36469b7e2d500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } } }
I20260807 19:40:52.310668 29406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:52.311251 29405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "46a72cb93f2543c595d36469b7e2d500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } } }
I20260807 19:40:52.311586 29405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:52.319774 29410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:40:52.330098 29410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:40:52.332535 29410 catalog_manager.cc:1259] Loaded cluster ID: fc3fceb082bc4e588cdd6312e2166472
I20260807 19:40:52.332638 29410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:40:52.336078 29410 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 19:40:52.354192 29410 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 19:40:52.380650 29410 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:40:52.382607 29410 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Loaded TSK: 0
I20260807 19:40:52.384227 29410 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:40:52.621241 29402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:52.621577 29402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:52.621749 29402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:52.628995 29402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:52.629149 29402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:52.629321 29402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:40:52.639106 29402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:44949
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=44273
--tserver_master_addrs=127.14.16.126:45053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:52.640506 29402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:52.641858 29402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:52.655475 29429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:52.655436 29430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:52.656970 29432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:52.657496 29402 server_base.cc:1034] running on GCE node
I20260807 19:40:52.658139 29402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:52.659173 29402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:52.660398 29402 hybrid_clock.cc:648] HybridClock initialized: now 1786131652660343 us; error 53 us; skew 500 ppm
I20260807 19:40:52.660708 29402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:52 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131652, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:52.673390 29402 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:40:52.675109 29402 webserver.cc:460] Webserver started at http://127.14.16.65:44273/ using document root <none> and password file <none>
I20260807 19:40:52.675908 29402 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:52.676076 29402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:52.681512 29402 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:40:52.684253 29439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:52.685621 29402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:40:52.685803 29402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "d186475b246e438a864cc5e6b6248aa0"
format_stamp: "Formatted at 2026-08-07 19:40:49 on dist-test-slave-07c2"
server_key: "b019ce05a9243a6bd803717ec1a6df80"
server_key_iv: "01239ac28b657747adec04df165c7a5d"
server_key_version: "encryptionkey@0"
I20260807 19:40:52.686762 29402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:52.720533 29402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:52.724915 29402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:52.725207 29402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:52.726225 29402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:52.727598 29402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:52.727725 29402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:52.727857 29402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:52.727962 29402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:52.763000 29402 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:44949
I20260807 19:40:52.763142 29552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:44949 every 8 connection(s)
I20260807 19:40:52.764484 29402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:40:52.771015 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 29402
I20260807 19:40:52.771766 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:33275
--local_ip_for_outbound_sockets=127.14.16.66
--tserver_master_addrs=127.14.16.126:45053
--webserver_port=43473
--webserver_interface=127.14.16.66
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:40:52 dist-test-slave-07c2 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131652, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:52.791961 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:52.767384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:40627 (local address 127.14.16.126:45053)
0807 19:40:52.767943 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:52.767953 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:52.768727 (+ 774us) server_negotiation.cc:407] Connection header received
0807 19:40:52.771056 (+ 2329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:52.771068 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:52.771502 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:52.771823 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:52.773851 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:52.775068 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:52.777606 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:52.778222 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:52.778364 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:52.783018 (+ 4654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:52.783031 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:52.783051 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:52.783102 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:52.787111 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:52.787896 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:52.787908 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:52.787918 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:52.788160 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:52.788588 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:52.788596 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:52.788602 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:52.789281 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:52.789485 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:52.790241 (+ 756us) server_negotiation.cc:299] Negotiation successful
0807 19:40:52.790637 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":259,"threads_started":1}
I20260807 19:40:52.791958 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:52.767655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:40627)
0807 19:40:52.768530 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:52.768589 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:52.770485 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:52.772036 (+ 1551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:52.772048 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:52.772844 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:52.773613 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:52.773639 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:52.775258 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:52.775266 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:52.777449 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:52.777468 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:52.778457 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:52.778466 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:52.778790 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:52.779581 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:52.779622 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:52.782685 (+ 3063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:52.787257 (+ 4572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:52.787264 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:52.787276 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:52.787740 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:52.788283 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:52.788286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:52.788288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:52.788460 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:52.789416 (+ 956us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:52.789422 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:52.789719 (+ 297us) client_negotiation.cc:769] Sending connection context
0807 19:40:52.789992 (+ 273us) client_negotiation.cc:240] Negotiation successful
0807 19:40:52.790574 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":395,"threads_started":1}
I20260807 19:40:52.795112 29553 heartbeater.cc:347] Connected to a master server at 127.14.16.126:45053
I20260807 19:40:52.795451 29553 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:52.796336 29553 heartbeater.cc:511] Master 127.14.16.126:45053 requested a full tablet report, sending...
I20260807 19:40:52.798872 29364 ts_manager.cc:194] Registered new tserver with Master: d186475b246e438a864cc5e6b6248aa0 (127.14.16.65:44949)
I20260807 19:40:52.801210 29364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:40627
W20260807 19:40:53.119982 29557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:53.120275 29557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:53.120373 29557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:53.127553 29557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:53.127664 29557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:53.127779 29557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:40:53.137380 29557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:33275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=43473
--tserver_master_addrs=127.14.16.126:45053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:53.138715 29557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:53.140066 29557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:53.152812 29562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:53.152838 29565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:53.152872 29563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:53.153721 29557 server_base.cc:1034] running on GCE node
I20260807 19:40:53.154287 29557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:53.155198 29557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:53.156442 29557 hybrid_clock.cc:648] HybridClock initialized: now 1786131653156381 us; error 52 us; skew 500 ppm
I20260807 19:40:53.156812 29557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:53 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131653, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:53.160117 29557 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:40:53.161593 29557 webserver.cc:460] Webserver started at http://127.14.16.66:43473/ using document root <none> and password file <none>
I20260807 19:40:53.162428 29557 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:53.162618 29557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:53.167968 29557 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:40:53.170663 29572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:53.172169 29557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:40:53.172350 29557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "9d976c31385749c9b56afd3b415ae94d"
format_stamp: "Formatted at 2026-08-07 19:40:50 on dist-test-slave-07c2"
server_key: "640cbb75eb177fc5924ce2c1e1fab09e"
server_key_iv: "acc501084904d5184f208fac428f1ff9"
server_key_version: "encryptionkey@0"
I20260807 19:40:53.173326 29557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:53.187944 29557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:53.192291 29557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:53.192569 29557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:53.193658 29557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:53.194993 29557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:53.195106 29557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:53.195235 29557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:53.195345 29557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:53.230178 29557 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:33275
I20260807 19:40:53.230273 29688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:33275 every 8 connection(s)
I20260807 19:40:53.231671 29557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:40:53.234320 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 29557
I20260807 19:40:53.235090 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:44471
--local_ip_for_outbound_sockets=127.14.16.67
--tserver_master_addrs=127.14.16.126:45053
--webserver_port=44351
--webserver_interface=127.14.16.67
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:40:53 dist-test-slave-07c2 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131653, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:53.255869 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.234767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:57215 (local address 127.14.16.126:45053)
0807 19:40:53.234973 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:53.234980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:53.236155 (+ 1175us) server_negotiation.cc:407] Connection header received
0807 19:40:53.238816 (+ 2661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:53.238825 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:53.238951 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:53.239073 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:53.241004 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.242173 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.244486 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.245282 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.245426 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.249456 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:53.249461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:53.249464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:53.249496 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:53.252568 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:53.253350 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:53.253354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:53.253357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:53.253507 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:53.253973 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:53.253976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:53.253979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:53.254266 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:53.254433 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:53.255200 (+ 767us) server_negotiation.cc:299] Negotiation successful
0807 19:40:53.255488 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:40:53.256920 29691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.234933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:57215)
0807 19:40:53.235969 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:53.236019 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:53.238481 (+ 2462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:53.239370 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:53.239381 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:53.240106 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:53.240802 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.240825 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.242349 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.242355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.244320 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.244335 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.245522 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.245528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.245793 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.246605 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:53.246633 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:53.249234 (+ 2601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:53.252733 (+ 3499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:53.252744 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:53.252761 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:53.253206 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:53.253624 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:53.253629 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:53.253632 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:53.253836 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:53.254402 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:53.254410 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:53.254791 (+ 381us) client_negotiation.cc:769] Sending connection context
0807 19:40:53.255067 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 19:40:53.255649 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":497,"threads_started":1}
I20260807 19:40:53.258409 29689 heartbeater.cc:347] Connected to a master server at 127.14.16.126:45053
I20260807 19:40:53.258744 29689 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:53.259565 29689 heartbeater.cc:511] Master 127.14.16.126:45053 requested a full tablet report, sending...
I20260807 19:40:53.261111 29364 ts_manager.cc:194] Registered new tserver with Master: 9d976c31385749c9b56afd3b415ae94d (127.14.16.66:33275)
I20260807 19:40:53.262099 29364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:57215
W20260807 19:40:53.586815 29692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:53.587118 29692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:53.587216 29692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:53.594717 29692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:53.594848 29692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:53.594965 29692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:40:53.605157 29692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:44471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=44351
--tserver_master_addrs=127.14.16.126:45053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:53.606498 29692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:53.607867 29692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:53.620890 29697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:53.620822 29700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:53.620867 29698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:53.622129 29692 server_base.cc:1034] running on GCE node
I20260807 19:40:53.622669 29692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:53.623584 29692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:53.624792 29692 hybrid_clock.cc:648] HybridClock initialized: now 1786131653624730 us; error 61 us; skew 500 ppm
I20260807 19:40:53.625056 29692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:53 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131653, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:53.628058 29692 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:40:53.629532 29692 webserver.cc:460] Webserver started at http://127.14.16.67:44351/ using document root <none> and password file <none>
I20260807 19:40:53.630203 29692 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:53.630364 29692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:53.635515 29692 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:40:53.638103 29707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:53.639496 29692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:40:53.639675 29692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "1fe054d2c0b146098eacaf66274d2559"
format_stamp: "Formatted at 2026-08-07 19:40:50 on dist-test-slave-07c2"
server_key: "3b80a2f3f661c3af206982828c4b5f6b"
server_key_iv: "39648077cab60751db805b30f4a6cb37"
server_key_version: "encryptionkey@0"
I20260807 19:40:53.640617 29692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:53.655978 29692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:53.659912 29692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:53.660182 29692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:53.661232 29692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:53.662607 29692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:53.662737 29692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:53.662866 29692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:53.662961 29692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:53.697139 29692 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:44471
I20260807 19:40:53.697203 29824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:44471 every 8 connection(s)
I20260807 19:40:53.698657 29692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:40:53.700518 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 29692
Aug 07 19:40:53 dist-test-slave-07c2 krb5kdc[28757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131653, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:40:53.716427 29828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.701819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48076 (local address 127.14.16.126:45053)
0807 19:40:53.702271 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:53.702277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:53.702302 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:40:53.702448 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:53.702451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:53.702513 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:53.702604 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:53.704782 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.705922 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.707314 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.707937 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.708087 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.710778 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:53.710784 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:53.710789 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:53.710825 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:53.713531 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:53.714311 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:53.714318 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:53.714322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:53.714451 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:53.714860 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:53.714863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:53.714865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:53.715119 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:53.715438 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:53.715757 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 19:40:53.715978 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":352,"threads_started":1}
I20260807 19:40:53.721544 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.701240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:47547 (local address 127.14.16.126:45053)
0807 19:40:53.701439 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:53.701445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:53.703956 (+ 2511us) server_negotiation.cc:407] Connection header received
0807 19:40:53.706245 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:53.706250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:53.706324 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:53.706439 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:53.708127 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.708991 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.710546 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.711256 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.711416 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.715306 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:53.715314 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:53.715319 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:53.715351 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:53.718918 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:53.719698 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:53.719702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:53.719704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:53.719814 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:53.720203 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:53.720205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:53.720207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:53.720443 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:53.720547 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:53.721171 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 19:40:53.721316 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:40:53.722481 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.701739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:47547)
0807 19:40:53.703799 (+ 2060us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:53.703836 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:53.705974 (+ 2138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:53.706676 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:53.706684 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:53.707153 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:40:53.707947 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.707972 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.709217 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.709225 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.710398 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.710409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.711495 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.711500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.711834 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.712499 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:40:53.712533 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:40:53.715080 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:40:53.719053 (+ 3973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:53.719063 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:53.719078 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:53.719591 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:53.719911 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:40:53.719914 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:40:53.719916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:40:53.720095 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:40:53.720546 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:40:53.720551 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:40:53.720853 (+ 302us) client_negotiation.cc:769] Sending connection context
0807 19:40:53.721070 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 19:40:53.721431 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1776,"thread_start_us":1574,"threads_started":1}
I20260807 19:40:53.723894 29825 heartbeater.cc:347] Connected to a master server at 127.14.16.126:45053
I20260807 19:40:53.724201 29825 heartbeater.cc:464] Registering TS with master...
I20260807 19:40:53.724913 29825 heartbeater.cc:511] Master 127.14.16.126:45053 requested a full tablet report, sending...
I20260807 19:40:53.726207 29364 ts_manager.cc:194] Registered new tserver with Master: 1fe054d2c0b146098eacaf66274d2559 (127.14.16.67:44471)
I20260807 19:40:53.727013 29364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:47547
I20260807 19:40:53.731477 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:40:53.738646 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.732472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48084 (local address 127.14.16.126:45053)
0807 19:40:53.732677 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:53.732685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:53.733071 (+ 386us) server_negotiation.cc:407] Connection header received
0807 19:40:53.733305 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:53.733311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:53.733460 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:53.733618 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:40:53.734066 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.734834 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.736476 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.736892 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.737026 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.737561 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:40:53.738038 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:40:53.738245 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:53.738294 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 19:40:53.738375 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:40:53.739748 29364 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:40:53.747045 29364 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:40:53.779213 29837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.758591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35072 (local address 127.14.16.66:33275)
0807 19:40:53.759161 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:53.759169 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:53.759907 (+ 738us) server_negotiation.cc:407] Connection header received
0807 19:40:53.760440 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:53.760451 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:53.760704 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:53.760879 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:53.761995 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.763964 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.776399 (+ 12435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.777919 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.778079 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.778280 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:53.778639 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 19:40:53.778846 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":296,"threads_started":1}
I20260807 19:40:53.780061 29836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.758425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:33275 (local address 127.0.0.1:35072)
0807 19:40:53.759773 (+ 1348us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:53.759808 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:53.760051 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:53.761235 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:53.761242 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:53.761285 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:53.761752 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.761770 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.773556 (+ 11786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.773561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.776141 (+ 2580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.776157 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.778145 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.778151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.778371 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.778434 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 19:40:53.779519 (+ 1085us) client_negotiation.cc:240] Negotiation successful
0807 19:40:53.779717 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":1,"thread_start_us":12526,"threads_started":1}
I20260807 19:40:53.781595 29835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.758561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:44949 (local address 127.0.0.1:54056)
0807 19:40:53.770678 (+ 12117us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:53.770705 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:53.770875 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:53.773843 (+ 2968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:53.773850 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:53.773876 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:53.774273 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.774288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.776594 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.776600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.778776 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.778792 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.780721 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.780736 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.780936 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.780975 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 19:40:53.781070 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 19:40:53.781218 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11941,"mutex_wait_us":24,"thread_start_us":850,"threads_started":1}
I20260807 19:40:53.781651 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.759161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54056 (local address 127.14.16.65:44949)
0807 19:40:53.772971 (+ 13810us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:53.772982 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:53.773013 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:40:53.773113 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:53.773121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:53.773372 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:53.773556 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:53.774897 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.776442 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.778939 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.780482 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.780636 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.780850 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:53.781136 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 19:40:53.781296 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13621,"thread_start_us":13446,"threads_started":1}
I20260807 19:40:53.784155 29622 tablet_service.cc:1467] Processing CreateTablet for tablet aab55aa05c7246dc9a491cfdc579787b (DEFAULT_TABLE table=test-table [id=235b352f6fc948ca852ceda6c3618f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:40:53.784830 29839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.760294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:44471 (local address 127.0.0.1:47544)
0807 19:40:53.773233 (+ 12939us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:53.773250 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:53.773361 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:53.776941 (+ 3580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:53.776945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:53.776978 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:53.777413 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.777428 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.780066 (+ 2638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.780072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.782299 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.782313 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.784079 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.784086 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.784259 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.784292 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:40:53.784396 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:40:53.784516 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12803,"thread_start_us":12669,"threads_started":1}
I20260807 19:40:53.784999 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.761081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47544 (local address 127.14.16.67:44471)
0807 19:40:53.776361 (+ 15280us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:53.776369 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:53.776404 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:40:53.776507 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:53.776514 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:53.776736 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:53.776947 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:53.777660 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.779909 (+ 2249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.782470 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.783852 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.783982 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.784185 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:53.784469 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 19:40:53.784644 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15095,"thread_start_us":14919,"threads_started":1}
I20260807 19:40:53.784888 29488 tablet_service.cc:1467] Processing CreateTablet for tablet aab55aa05c7246dc9a491cfdc579787b (DEFAULT_TABLE table=test-table [id=235b352f6fc948ca852ceda6c3618f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:40:53.785943 29622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aab55aa05c7246dc9a491cfdc579787b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:53.786208 29488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aab55aa05c7246dc9a491cfdc579787b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:53.788635 29758 tablet_service.cc:1467] Processing CreateTablet for tablet aab55aa05c7246dc9a491cfdc579787b (DEFAULT_TABLE table=test-table [id=235b352f6fc948ca852ceda6c3618f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:40:53.789991 29758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aab55aa05c7246dc9a491cfdc579787b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:40:53.802037 29842 tablet_bootstrap.cc:492] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0: Bootstrap starting.
I20260807 19:40:53.804387 29553 heartbeater.cc:503] Master 127.14.16.126:45053 was elected leader, sending a full tablet report...
I20260807 19:40:53.804795 29843 tablet_bootstrap.cc:492] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559: Bootstrap starting.
I20260807 19:40:53.807484 29842 tablet_bootstrap.cc:654] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:53.808192 29843 tablet_bootstrap.cc:654] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:53.808923 29842 log.cc:824] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:53.809352 29843 log.cc:824] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:53.809790 29841 tablet_bootstrap.cc:492] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d: Bootstrap starting.
I20260807 19:40:53.812508 29842 tablet_bootstrap.cc:492] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0: No bootstrap required, opened a new log
I20260807 19:40:53.812815 29842 ts_tablet_manager.cc:1397] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260807 19:40:53.813477 29843 tablet_bootstrap.cc:492] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559: No bootstrap required, opened a new log
I20260807 19:40:53.813736 29843 ts_tablet_manager.cc:1397] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260807 19:40:53.813786 29841 tablet_bootstrap.cc:654] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d: Neither blocks nor log segments found. Creating new log.
I20260807 19:40:53.815027 29841 log.cc:824] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:53.821888 29841 tablet_bootstrap.cc:492] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d: No bootstrap required, opened a new log
I20260807 19:40:53.821771 29843 raft_consensus.cc:348] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:53.822170 29841 ts_tablet_manager.cc:1397] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260807 19:40:53.822199 29843 raft_consensus.cc:374] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:53.822372 29843 raft_consensus.cc:729] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe054d2c0b146098eacaf66274d2559, State: Initialized, Role: FOLLOWER
I20260807 19:40:53.822985 29843 consensus_queue.cc:260] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:53.824097 29825 heartbeater.cc:503] Master 127.14.16.126:45053 was elected leader, sending a full tablet report...
I20260807 19:40:53.824126 29842 raft_consensus.cc:348] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:53.824523 29842 raft_consensus.cc:374] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:53.824630 29842 raft_consensus.cc:729] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d186475b246e438a864cc5e6b6248aa0, State: Initialized, Role: FOLLOWER
I20260807 19:40:53.824749 29843 ts_tablet_manager.cc:1428] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260807 19:40:53.825177 29842 consensus_queue.cc:260] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:53.829093 29842 ts_tablet_manager.cc:1428] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0: Time spent starting tablet: real 0.016s user 0.009s sys 0.007s
I20260807 19:40:53.833161 29841 raft_consensus.cc:348] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:53.833600 29841 raft_consensus.cc:374] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:40:53.833705 29841 raft_consensus.cc:729] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d976c31385749c9b56afd3b415ae94d, State: Initialized, Role: FOLLOWER
I20260807 19:40:53.834272 29841 consensus_queue.cc:260] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:53.835680 29689 heartbeater.cc:503] Master 127.14.16.126:45053 was elected leader, sending a full tablet report...
I20260807 19:40:53.836301 29841 ts_tablet_manager.cc:1428] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260807 19:40:53.910032 29849 raft_consensus.cc:482] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:40:53.910380 29849 raft_consensus.cc:504] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:53.912089 29849 leader_election.cc:288] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d186475b246e438a864cc5e6b6248aa0 (127.14.16.65:44949), 1fe054d2c0b146098eacaf66274d2559 (127.14.16.67:44471)
I20260807 19:40:53.920504 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.912666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:40971 (local address 127.14.16.65:44949)
0807 19:40:53.912930 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:53.912937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:53.913286 (+ 349us) server_negotiation.cc:407] Connection header received
0807 19:40:53.913578 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:53.913586 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:53.913698 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:53.913806 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:53.914490 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.915574 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.918227 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.919479 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.919648 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.919734 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:53.920131 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 19:40:53.920240 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:40:53.920717 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.913475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:60429 (local address 127.14.16.67:44471)
0807 19:40:53.913681 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:53.913689 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:53.914201 (+ 512us) server_negotiation.cc:407] Connection header received
0807 19:40:53.914383 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:53.914387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:53.914484 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:53.914588 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:40:53.915082 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.916018 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.918410 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.919709 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.919860 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.919908 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:53.920346 (+ 438us) server_negotiation.cc:299] Negotiation successful
0807 19:40:53.920470 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 19:40:53.920722 29850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.912460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:44949 (local address 127.14.16.66:40971)
0807 19:40:53.913174 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:53.913190 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:53.913380 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:53.913858 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:53.913867 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:53.913896 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:53.914331 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.914344 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.915708 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.915711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.918060 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.918075 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.919701 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.919705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.919902 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.919987 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 19:40:53.920092 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 19:40:53.920235 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":156,"thread_start_us":383,"threads_started":1}
I20260807 19:40:53.920796 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.913304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:44471 (local address 127.14.16.66:60429)
0807 19:40:53.914066 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:53.914094 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:40:53.914241 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:40:53.914604 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:40:53.914607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:40:53.914627 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:40:53.914918 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.914925 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.916154 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.916159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.918271 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:40:53.918281 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.919917 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.919923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:40:53.920186 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.920218 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:40:53.920303 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 19:40:53.920426 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":45,"thread_start_us":365,"threads_started":1}
I20260807 19:40:53.921998 29778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab55aa05c7246dc9a491cfdc579787b" candidate_uuid: "9d976c31385749c9b56afd3b415ae94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe054d2c0b146098eacaf66274d2559" is_pre_election: true
I20260807 19:40:53.921994 29508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab55aa05c7246dc9a491cfdc579787b" candidate_uuid: "9d976c31385749c9b56afd3b415ae94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d186475b246e438a864cc5e6b6248aa0" is_pre_election: true
I20260807 19:40:53.922605 29508 raft_consensus.cc:2454] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d976c31385749c9b56afd3b415ae94d in term 0.
I20260807 19:40:53.922605 29778 raft_consensus.cc:2454] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d976c31385749c9b56afd3b415ae94d in term 0.
I20260807 19:40:53.923384 29574 leader_election.cc:302] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d976c31385749c9b56afd3b415ae94d, d186475b246e438a864cc5e6b6248aa0; no voters:
I20260807 19:40:53.923818 29849 raft_consensus.cc:2781] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:40:53.923972 29849 raft_consensus.cc:482] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:40:53.924089 29849 raft_consensus.cc:3037] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:53.926319 29849 raft_consensus.cc:504] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:53.926918 29849 leader_election.cc:288] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [CANDIDATE]: Term 1 election: Requested vote from peers d186475b246e438a864cc5e6b6248aa0 (127.14.16.65:44949), 1fe054d2c0b146098eacaf66274d2559 (127.14.16.67:44471)
I20260807 19:40:53.927398 29508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab55aa05c7246dc9a491cfdc579787b" candidate_uuid: "9d976c31385749c9b56afd3b415ae94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d186475b246e438a864cc5e6b6248aa0"
I20260807 19:40:53.927456 29778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab55aa05c7246dc9a491cfdc579787b" candidate_uuid: "9d976c31385749c9b56afd3b415ae94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe054d2c0b146098eacaf66274d2559"
I20260807 19:40:53.927613 29508 raft_consensus.cc:3037] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:53.927670 29778 raft_consensus.cc:3037] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:40:53.930011 29778 raft_consensus.cc:2454] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d976c31385749c9b56afd3b415ae94d in term 1.
I20260807 19:40:53.930081 29508 raft_consensus.cc:2454] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d976c31385749c9b56afd3b415ae94d in term 1.
I20260807 19:40:53.930438 29574 leader_election.cc:302] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe054d2c0b146098eacaf66274d2559, 9d976c31385749c9b56afd3b415ae94d; no voters:
I20260807 19:40:53.930773 29849 raft_consensus.cc:2781] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:40:53.931391 29849 raft_consensus.cc:686] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 1 LEADER]: Becoming Leader. State: Replica: 9d976c31385749c9b56afd3b415ae94d, State: Running, Role: LEADER
I20260807 19:40:53.931895 29849 consensus_queue.cc:237] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } }
I20260807 19:40:53.936787 29364 catalog_manager.cc:5495] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d976c31385749c9b56afd3b415ae94d (127.14.16.66). New cstate: current_term: 1 leader_uuid: "9d976c31385749c9b56afd3b415ae94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d976c31385749c9b56afd3b415ae94d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 } health_report { overall_health: UNKNOWN } } }
W20260807 19:40:53.950269 29826 tablet.cc:2369] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:40:53.981277 29837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:53.975656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35082 (local address 127.14.16.66:33275)
0807 19:40:53.975895 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:53.975902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:53.975924 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:40:53.976064 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:53.976068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:53.976148 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:53.976305 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:40:53.976699 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.977512 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.979001 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:53.979493 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:53.979625 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:53.980164 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:40:53.980665 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:40:53.980842 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:53.980897 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 19:40:53.980982 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 19:40:53.983588 29690 tablet.cc:2369] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:40:53.990082 29778 raft_consensus.cc:1261] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559 [term 1 FOLLOWER]: Refusing update from remote peer 9d976c31385749c9b56afd3b415ae94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:40:53.990260 29508 raft_consensus.cc:1261] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0 [term 1 FOLLOWER]: Refusing update from remote peer 9d976c31385749c9b56afd3b415ae94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:40:53.991163 29852 consensus_queue.cc:1035] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fe054d2c0b146098eacaf66274d2559" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:40:53.991623 29849 consensus_queue.cc:1035] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d186475b246e438a864cc5e6b6248aa0" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 44949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:40:54.004880 29857 mvcc.cc:205] Tried to move back new op lower bound from 7315995254732660736 to 7315995254508269568. Current Snapshot: MvccSnapshot[applied={T|T < 7315995254732660736}]
I20260807 19:40:54.020080 29363 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:40:54.020378 29363 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:40:54.024106 29363 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Sending DeleteTablet for 3 replicas of tablet aab55aa05c7246dc9a491cfdc579787b
I20260807 19:40:54.025318 29622 tablet_service.cc:1514] Processing DeleteTablet for tablet aab55aa05c7246dc9a491cfdc579787b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:40:54 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:35072
I20260807 19:40:54.025529 29488 tablet_service.cc:1514] Processing DeleteTablet for tablet aab55aa05c7246dc9a491cfdc579787b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:40:54 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:54056
I20260807 19:40:54.025537 29758 tablet_service.cc:1514] Processing DeleteTablet for tablet aab55aa05c7246dc9a491cfdc579787b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:40:54 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:47544
I20260807 19:40:54.026561 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 29402
I20260807 19:40:54.037997 29867 tablet_replica.cc:330] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0: stopping tablet replica
I20260807 19:40:54.038780 29867 raft_consensus.cc:2229] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:40:54.039577 29868 tablet_replica.cc:330] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559: stopping tablet replica
I20260807 19:40:54.039602 29867 raft_consensus.cc:2258] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:40:54.040436 29868 raft_consensus.cc:2229] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:40:54.040984 29868 raft_consensus.cc:2258] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:40:54.041143 29866 tablet_replica.cc:330] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d: stopping tablet replica
I20260807 19:40:54.041618 29866 raft_consensus.cc:2229] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:40:54.042336 29867 ts_tablet_manager.cc:1905] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:40:54.042491 29866 raft_consensus.cc:2258] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:40:54.044677 29866 ts_tablet_manager.cc:1905] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:40:54.048421 29868 ts_tablet_manager.cc:1905] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:40:54.049330 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:54.027436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54068 (local address 127.14.16.65:44949)
0807 19:40:54.027654 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:54.027662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:54.027687 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:40:54.027872 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:54.027880 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:54.027991 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:54.028126 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:54.030498 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:54.032075 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:54.034623 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:54.035589 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:54.035745 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:54.040185 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:54.040205 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:54.040225 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:54.040269 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:54.044648 (+ 4379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:54.045760 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:54.045769 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:54.045772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:54.045946 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:54.046452 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:54.046457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:54.046462 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:54.046955 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:54.047312 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:54.048662 (+ 1350us) server_negotiation.cc:299] Negotiation successful
0807 19:40:54.048928 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:40:54.050239 29548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:54.052142 29866 ts_tablet_manager.cc:1918] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:40:54.052357 29866 log.cc:1196] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/aab55aa05c7246dc9a491cfdc579787b
I20260807 19:40:54.053004 29866 ts_tablet_manager.cc:1939] T aab55aa05c7246dc9a491cfdc579787b P 9d976c31385749c9b56afd3b415ae94d: Deleting consensus metadata
I20260807 19:40:54.054843 29350 catalog_manager.cc:4841] TS 9d976c31385749c9b56afd3b415ae94d (127.14.16.66:33275): tablet aab55aa05c7246dc9a491cfdc579787b (table test-table [id=235b352f6fc948ca852ceda6c3618f58]) successfully deleted
I20260807 19:40:54.055440 29868 ts_tablet_manager.cc:1918] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:40:54.055859 29868 log.cc:1196] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/aab55aa05c7246dc9a491cfdc579787b
I20260807 19:40:54.056933 29868 ts_tablet_manager.cc:1939] T aab55aa05c7246dc9a491cfdc579787b P 1fe054d2c0b146098eacaf66274d2559: Deleting consensus metadata
I20260807 19:40:54.059188 29349 catalog_manager.cc:4841] TS 1fe054d2c0b146098eacaf66274d2559 (127.14.16.67:44471): tablet aab55aa05c7246dc9a491cfdc579787b (table test-table [id=235b352f6fc948ca852ceda6c3618f58]) successfully deleted
I20260807 19:40:54.147044 29867 ts_tablet_manager.cc:1918] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:40:54.147274 29867 log.cc:1196] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/aab55aa05c7246dc9a491cfdc579787b
I20260807 19:40:54.147934 29867 ts_tablet_manager.cc:1939] T aab55aa05c7246dc9a491cfdc579787b P d186475b246e438a864cc5e6b6248aa0: Deleting consensus metadata
I20260807 19:40:54.150007 29349 catalog_manager.cc:4841] TS d186475b246e438a864cc5e6b6248aa0 (127.14.16.65:44949): tablet aab55aa05c7246dc9a491cfdc579787b (table test-table [id=235b352f6fc948ca852ceda6c3618f58]) successfully deleted
I20260807 19:40:54.150416 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 29402
I20260807 19:40:54.170929 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 29557
I20260807 19:40:54.187067 29837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:54.171792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35096 (local address 127.14.16.66:33275)
0807 19:40:54.171993 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:54.171998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:54.172211 (+ 213us) server_negotiation.cc:407] Connection header received
0807 19:40:54.172368 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:54.172374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:54.172481 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:54.172606 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:54.173977 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:54.175242 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:54.176789 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:54.177459 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:54.177608 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:54.180490 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:54.180509 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:54.180529 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:54.180582 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:54.184072 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:54.184804 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:54.184815 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:54.184818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:54.184958 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:54.185363 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:54.185369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:54.185372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:54.185753 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:54.186098 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:54.186412 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 19:40:54.186640 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:40:54.187850 29684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:54.303982 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 29557
I20260807 19:40:54.326478 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 29692
I20260807 19:40:54.340659 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:54.327328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47558 (local address 127.14.16.67:44471)
0807 19:40:54.327523 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 19:40:54.327529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:40:54.327649 (+ 120us) server_negotiation.cc:407] Connection header received
0807 19:40:54.327876 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:40:54.327880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:40:54.327945 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:40:54.328078 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:40:54.329536 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:54.330254 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:54.331742 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:40:54.332131 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:40:54.332270 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:40:54.334918 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:40:54.334933 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:40:54.334949 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:40:54.334983 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:40:54.337665 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:54.338410 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:54.338418 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:54.338420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:54.338561 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:40:54.338995 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:40:54.339000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:40:54.339003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:40:54.339454 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:40:54.339772 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 19:40:54.340079 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 19:40:54.340304 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:40:54.341375 29820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:54.431799 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 29692
I20260807 19:40:54.451009 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 29331
I20260807 19:40:54.452332 29395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:40:54.567962 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 29331
I20260807 19:40:54.602319 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:45053
--webserver_interface=127.14.16.126
--webserver_port=36857
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:45053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:40:54.941612 29874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:54.941946 29874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:54.942112 29874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:54.950057 29874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:40:54.950201 29874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:54.950301 29874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:54.950402 29874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:40:54.950477 29874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:40:54.960516 29874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:45053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:45053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=36857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:54.961880 29874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:54.963356 29874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:54.976084 29882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:54.976125 29879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:54.976176 29880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:54.980835 29874 server_base.cc:1034] running on GCE node
I20260807 19:40:54.981717 29874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:54.983187 29874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:54.984390 29874 hybrid_clock.cc:648] HybridClock initialized: now 1786131654984347 us; error 34 us; skew 500 ppm
I20260807 19:40:54.984781 29874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:54 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131654, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:54.987949 29874 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:40:54.989549 29874 webserver.cc:460] Webserver started at http://127.14.16.126:36857/ using document root <none> and password file <none>
I20260807 19:40:54.990278 29874 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:54.990428 29874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:54.995849 29874 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:40:54.998621 29889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:55.000023 29874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:40:55.000222 29874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "46a72cb93f2543c595d36469b7e2d500"
format_stamp: "Formatted at 2026-08-07 19:40:49 on dist-test-slave-07c2"
server_key: "6b0c161815344c79a18615565b48a31b"
server_key_iv: "0146d381f9316b621ee911ec649e2fe4"
server_key_version: "encryptionkey@0"
I20260807 19:40:55.001181 29874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:55.027504 29874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:55.031766 29874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:55.032058 29874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:55.051469 29874 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:45053
I20260807 19:40:55.051445 29941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:45053 every 8 connection(s)
I20260807 19:40:55.053002 29874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:40:55.059888 29942 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:40:55.061141 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 29874
I20260807 19:40:55.062024 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:44949
--local_ip_for_outbound_sockets=127.14.16.65
--tserver_master_addrs=127.14.16.126:45053
--webserver_port=44273
--webserver_interface=127.14.16.65
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:40:55.063566 29942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Bootstrap starting.
I20260807 19:40:55.084412 29942 log.cc:824] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Log is configured to *not* fsync() on all Append() calls
I20260807 19:40:55.101545 29942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:40:55.102128 29942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500: Bootstrap complete.
I20260807 19:40:55.112054 29942 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } }
I20260807 19:40:55.113020 29942 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a72cb93f2543c595d36469b7e2d500, State: Initialized, Role: FOLLOWER
I20260807 19:40:55.113718 29942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } }
I20260807 19:40:55.113973 29942 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:40:55.114114 29942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:40:55.114318 29942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:40:55.118248 29942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } }
I20260807 19:40:55.118764 29942 leader_election.cc:302] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46a72cb93f2543c595d36469b7e2d500; no voters:
I20260807 19:40:55.119498 29942 leader_election.cc:288] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 19:40:55.119647 29946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:40:55.120939 29946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [term 3 LEADER]: Becoming Leader. State: Replica: 46a72cb93f2543c595d36469b7e2d500, State: Running, Role: LEADER
I20260807 19:40:55.121547 29946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } }
I20260807 19:40:55.121845 29942 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:40:55.124527 29948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46a72cb93f2543c595d36469b7e2d500. Latest consensus state: current_term: 3 leader_uuid: "46a72cb93f2543c595d36469b7e2d500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } } }
I20260807 19:40:55.124966 29948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:55.125483 29947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "46a72cb93f2543c595d36469b7e2d500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a72cb93f2543c595d36469b7e2d500" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 45053 } } }
I20260807 19:40:55.125821 29947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a72cb93f2543c595d36469b7e2d500 [sys.catalog]: This master's current role is: LEADER
I20260807 19:40:55.137107 29952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:40:55.146076 29952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:40:55.147631 29952 catalog_manager.cc:1259] Loaded cluster ID: fc3fceb082bc4e588cdd6312e2166472
I20260807 19:40:55.147804 29952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 19:40:55.149956 29952 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f34ec8729ad google::LogMessage::Fail() at ??:0
@ 0x7f34ec87588a google::LogMessage::SendToLog() at ??:0
@ 0x7f34ec8724de google::LogMessage::Flush() at ??:0
@ 0x7f34ec873a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f35022a5512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f35022a2976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f350232a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f350232a439 std::_Function_handler<>::_M_invoke() at ??:0
I20260807 19:40:55.189441 29967 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f34ef4be345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f34ef4c191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f34ef4c1799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f34ef49ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f34f27766db start_thread at ??:0
@ 0x7f34eb70c61f clone at ??:0
W20260807 19:40:55.473518 29944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:55.473802 29944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:55.473897 29944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:55.481489 29944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:55.481611 29944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:55.481731 29944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:40:55.492087 29944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:44949
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=44273
--tserver_master_addrs=127.14.16.126:45053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:55.493398 29944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:55.494772 29944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:55.508556 29986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:55.508654 29983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:55.508682 29984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:55.509455 29944 server_base.cc:1034] running on GCE node
I20260807 19:40:55.510021 29944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:55.511009 29944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:55.512223 29944 hybrid_clock.cc:648] HybridClock initialized: now 1786131655512171 us; error 27 us; skew 500 ppm
I20260807 19:40:55.512554 29944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:55 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131655, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:55.515728 29944 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:40:55.517216 29944 webserver.cc:460] Webserver started at http://127.14.16.65:44273/ using document root <none> and password file <none>
I20260807 19:40:55.517911 29944 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:55.518064 29944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:55.523636 29944 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:40:55.528349 29993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:55.529609 29944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:40:55.529785 29944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "d186475b246e438a864cc5e6b6248aa0"
format_stamp: "Formatted at 2026-08-07 19:40:49 on dist-test-slave-07c2"
server_key: "b019ce05a9243a6bd803717ec1a6df80"
server_key_iv: "01239ac28b657747adec04df165c7a5d"
server_key_version: "encryptionkey@0"
I20260807 19:40:55.530853 29944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:55.543529 29944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:55.547490 29944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:55.547773 29944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:55.548784 29944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:55.550137 29944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:55.550273 29944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:55.550396 29944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:55.550486 29944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:55.588975 29944 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:44949
I20260807 19:40:55.589064 30108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:44949 every 8 connection(s)
I20260807 19:40:55.590477 29944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:40:55.592200 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 29944
I20260807 19:40:55.592820 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:33275
--local_ip_for_outbound_sockets=127.14.16.66
--tserver_master_addrs=127.14.16.126:45053
--webserver_port=43473
--webserver_interface=127.14.16.66
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:40:55.595817 30112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:55.593483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:57891)
0807 19:40:55.594371 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:55.594668 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":303,"threads_started":1}
W20260807 19:40:55.596575 30109 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:45053 (0 consecutive failures): Network error: Failed to ping master at 127.14.16.126:45053: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
I20260807 19:40:55.597533 30112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:55.597100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:35315)
0807 19:40:55.597289 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:55.597382 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 19:40:55.598510 30112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:55.598088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:58769)
0807 19:40:55.598254 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:55.598355 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 19:40:55.598791 30109 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:40:55.930437 30113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:55.930721 30113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:55.930819 30113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:55.938251 30113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:55.938362 30113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:55.938476 30113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:40:55.948760 30113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:33275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=43473
--tserver_master_addrs=127.14.16.126:45053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:55.950016 30113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:55.951263 30113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:55.964406 30118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:55.964452 30121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:55.964452 30119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:55.965106 30113 server_base.cc:1034] running on GCE node
I20260807 19:40:55.965765 30113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:55.966662 30113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:55.967902 30113 hybrid_clock.cc:648] HybridClock initialized: now 1786131655967847 us; error 45 us; skew 500 ppm
I20260807 19:40:55.968276 30113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:55 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131655, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:55.971709 30113 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:40:55.973186 30113 webserver.cc:460] Webserver started at http://127.14.16.66:43473/ using document root <none> and password file <none>
I20260807 19:40:55.973909 30113 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:55.974064 30113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:55.979517 30113 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:40:55.982239 30128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:55.983610 30113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:40:55.983798 30113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "9d976c31385749c9b56afd3b415ae94d"
format_stamp: "Formatted at 2026-08-07 19:40:50 on dist-test-slave-07c2"
server_key: "640cbb75eb177fc5924ce2c1e1fab09e"
server_key_iv: "acc501084904d5184f208fac428f1ff9"
server_key_version: "encryptionkey@0"
I20260807 19:40:55.984822 30113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:56.003952 30113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:56.008140 30113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:56.008427 30113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:56.009482 30113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:56.010917 30113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:56.011047 30113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:56.011185 30113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:56.011277 30113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:56.045562 30113 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:33275
I20260807 19:40:56.045619 30242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:33275 every 8 connection(s)
I20260807 19:40:56.047158 30113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:40:56.051854 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 30113
I20260807 19:40:56.052150 30245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:56.050101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:43575)
0807 19:40:56.050810 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:56.051020 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":264,"threads_started":1}
I20260807 19:40:56.052640 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:44471
--local_ip_for_outbound_sockets=127.14.16.67
--tserver_master_addrs=127.14.16.126:45053
--webserver_port=44351
--webserver_interface=127.14.16.67
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:45257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:40:56.052867 30243 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:45053 (0 consecutive failures): Network error: Failed to ping master at 127.14.16.126:45053: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
I20260807 19:40:56.053826 30245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:56.053376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:50677)
0807 19:40:56.053565 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:56.053663 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 19:40:56.054816 30245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:56.054381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:58585)
0807 19:40:56.054547 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:56.054646 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 19:40:56.055112 30243 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:40:56.389668 30246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:40:56.389940 30246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:40:56.390196 30246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:40:56.397605 30246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:40:56.397708 30246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:40:56.397927 30246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:40:56.407523 30246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:44471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=44351
--tserver_master_addrs=127.14.16.126:45053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:40:56.408807 30246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:40:56.410182 30246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:40:56.423013 30254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:56.423069 30252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:40:56.423372 30251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:40:56.423799 30246 server_base.cc:1034] running on GCE node
I20260807 19:40:56.424384 30246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:40:56.425326 30246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:40:56.426576 30246 hybrid_clock.cc:648] HybridClock initialized: now 1786131656426537 us; error 48 us; skew 500 ppm
I20260807 19:40:56.426906 30246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:40:56 dist-test-slave-07c2 krb5kdc[28757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131656, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:40:56.430116 30246 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:40:56.431496 30246 webserver.cc:460] Webserver started at http://127.14.16.67:44351/ using document root <none> and password file <none>
I20260807 19:40:56.432119 30246 fs_manager.cc:359] Metadata directory not provided
I20260807 19:40:56.432220 30246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:40:56.437470 30246 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260807 19:40:56.440043 30261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:56.441324 30246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 19:40:56.441491 30246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "1fe054d2c0b146098eacaf66274d2559"
format_stamp: "Formatted at 2026-08-07 19:40:50 on dist-test-slave-07c2"
server_key: "3b80a2f3f661c3af206982828c4b5f6b"
server_key_iv: "39648077cab60751db805b30f4a6cb37"
server_key_version: "encryptionkey@0"
I20260807 19:40:56.442471 30246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:40:56.465464 30246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:40:56.469887 30246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:40:56.470182 30246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:40:56.471194 30246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:40:56.472540 30246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:40:56.472631 30246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:56.472711 30246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:40:56.472766 30246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:40:56.506588 30246 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:44471
I20260807 19:40:56.506716 30376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:44471 every 8 connection(s)
I20260807 19:40:56.508111 30246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:40:56.511881 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 30246
I20260807 19:40:56.513401 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:56.511090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:55421)
0807 19:40:56.512046 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:56.512361 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":377,"threads_started":1}
W20260807 19:40:56.514151 30377 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:45053 (0 consecutive failures): Network error: Failed to ping master at 127.14.16.126:45053: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
I20260807 19:40:56.514914 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:56.514562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:45271)
0807 19:40:56.514742 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:56.514811 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 19:40:56.515847 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:56.515516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:45095)
0807 19:40:56.515654 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:56.515743 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 19:40:56.516069 30377 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:40:56.600386 30381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:56.599532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:33613)
0807 19:40:56.600043 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:56.600154 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":255,"threads_started":1}
I20260807 19:40:57.056470 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:57.055679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:52885)
0807 19:40:57.056130 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:57.056247 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":215,"threads_started":1}
I20260807 19:40:57.517403 30383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:57.516627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:41079)
0807 19:40:57.517081 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:57.517174 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":214,"threads_started":1}
I20260807 19:40:57.602057 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:57.601318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:44235)
0807 19:40:57.601795 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:57.601883 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":226,"threads_started":1}
I20260807 19:40:58.058069 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:58.057375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:43891)
0807 19:40:58.057821 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:58.057903 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":223,"threads_started":1}
I20260807 19:40:58.519153 30386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:58.518397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:46249)
0807 19:40:58.518899 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:58.518986 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":247,"threads_started":1}
I20260807 19:40:58.603660 30387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:58.602927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:39771)
0807 19:40:58.603398 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:58.603486 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":252,"threads_started":1}
I20260807 19:40:59.059939 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:59.059064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:46597)
0807 19:40:59.059659 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:59.059768 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":332,"threads_started":1}
I20260807 19:40:59.520848 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:59.520110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:48501)
0807 19:40:59.520577 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:59.520659 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":260,"threads_started":1}
I20260807 19:40:59.605423 30390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:40:59.604597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:50467)
0807 19:40:59.605116 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 19:40:59.605205 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":263,"threads_started":1}
I20260807 19:41:00.061579 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:00.060813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:33457)
0807 19:41:00.061294 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:00.061384 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":234,"threads_started":1}
I20260807 19:41:00.522475 30392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:00.521735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:37455)
0807 19:41:00.522196 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:00.522278 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":232,"threads_started":1}
I20260807 19:41:00.607136 30393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:00.606411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:41479)
0807 19:41:00.606878 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:00.606962 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":246,"threads_started":1}
I20260807 19:41:01.063277 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:01.062526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:59741)
0807 19:41:01.062998 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:01.063084 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":247,"threads_started":1}
I20260807 19:41:01.524180 30395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:01.523456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:43979)
0807 19:41:01.523933 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:01.524013 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":256,"threads_started":1}
I20260807 19:41:01.608783 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:01.608056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:44243)
0807 19:41:01.608499 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:01.608604 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":227,"threads_started":1}
I20260807 19:41:02.064900 30397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:02.064190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:41985)
0807 19:41:02.064641 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:02.064737 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":240,"threads_started":1}
I20260807 19:41:02.525784 30399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:02.525054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:52861)
0807 19:41:02.525517 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:02.525605 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":260,"threads_started":1}
I20260807 19:41:02.610646 30400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:02.609795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:37527)
0807 19:41:02.610320 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:02.610415 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":226,"threads_started":1}
I20260807 19:41:03.066679 30401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:03.065893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:50905)
0807 19:41:03.066381 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:03.066464 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":218,"threads_started":1}
I20260807 19:41:03.527519 30402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:03.526661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:45765)
0807 19:41:03.527128 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:03.527231 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":207,"threads_started":1}
I20260807 19:41:03.612414 30403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:03.611703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:59751)
0807 19:41:03.612154 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:03.612246 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":229,"threads_started":1}
I20260807 19:41:04.068241 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:04.067554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:35731)
0807 19:41:04.067993 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:04.068075 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":222,"threads_started":1}
I20260807 19:41:04.529189 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:04.528426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:37847)
0807 19:41:04.528934 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:04.529021 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":276,"threads_started":1}
I20260807 19:41:04.614056 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:04.613312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:39707)
0807 19:41:04.613795 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:04.613879 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":226,"threads_started":1}
I20260807 19:41:05.069984 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:05.069246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:49959)
0807 19:41:05.069708 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:05.069806 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":218,"threads_started":1}
I20260807 19:41:05.531167 30408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:05.530119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:37839)
0807 19:41:05.530776 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:05.530887 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":343,"threads_started":1}
I20260807 19:41:05.615815 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:05.614989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:36735)
0807 19:41:05.615561 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:05.615646 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":361,"threads_started":1}
I20260807 19:41:06.071574 30410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:06.070848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:35895)
0807 19:41:06.071323 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:06.071413 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":237,"threads_started":1}
I20260807 19:41:06.532946 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:06.532209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:45563)
0807 19:41:06.532659 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:06.532747 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":236,"threads_started":1}
I20260807 19:41:06.617431 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:06.616682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:50433)
0807 19:41:06.617163 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:06.617247 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":232,"threads_started":1}
I20260807 19:41:07.073179 30413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:07.072434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:37703)
0807 19:41:07.072885 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:07.072974 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":231,"threads_started":1}
I20260807 19:41:07.534714 30414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:07.533908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:58581)
0807 19:41:07.534392 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:07.534488 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":258,"threads_started":1}
I20260807 19:41:07.619215 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:07.618457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:47751)
0807 19:41:07.618968 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:07.619056 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":284,"threads_started":1}
I20260807 19:41:08.074756 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:08.074068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:42251)
0807 19:41:08.074515 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:08.074597 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":229,"threads_started":1}
I20260807 19:41:08.536384 30417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:08.535706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:36005)
0807 19:41:08.536135 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:08.536224 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":221,"threads_started":1}
I20260807 19:41:08.620954 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:08.620178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:60237)
0807 19:41:08.620652 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:08.620773 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":227,"threads_started":1}
I20260807 19:41:09.076467 30419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:09.075711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:45981)
0807 19:41:09.076169 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:09.076278 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":223,"threads_started":1}
I20260807 19:41:09.538173 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:09.537250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:57003)
0807 19:41:09.537805 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:09.537923 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":256,"threads_started":1}
I20260807 19:41:09.622573 30421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:09.621852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:34927)
0807 19:41:09.622310 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:09.622391 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":226,"threads_started":1}
I20260807 19:41:10.078351 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:10.077617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:38775)
0807 19:41:10.078071 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:10.078161 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":238,"threads_started":1}
I20260807 19:41:10.539880 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:10.539119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:50183)
0807 19:41:10.539607 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:10.539695 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":283,"threads_started":1}
I20260807 19:41:10.624723 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:10.623849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.65:46739)
0807 19:41:10.624409 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:10.624510 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":272,"threads_started":1}
I20260807 19:41:11.080402 30425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:11.079531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.66:55083)
0807 19:41:11.080082 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:11.080196 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":293,"threads_started":1}
I20260807 19:41:11.513612 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 29944
I20260807 19:41:11.532829 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:11.515581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46380 (local address 127.14.16.65:44949)
0807 19:41:11.516234 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:11.516256 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:11.516312 (+ 56us) server_negotiation.cc:407] Connection header received
0807 19:41:11.518019 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:11.518030 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:11.518450 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:11.518742 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:11.520182 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:11.521524 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:11.522758 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:11.523516 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:11.523650 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:11.526656 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:11.526665 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:11.526686 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:11.526742 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:11.529654 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:11.530350 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:11.530357 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:11.530364 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:11.530537 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:11.530906 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:11.530909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:11.530911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:11.531545 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:11.531814 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:11.532173 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 19:41:11.532507 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":438,"threads_started":1}
I20260807 19:41:11.534282 30104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:11.541895 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:11.541050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:45053 (local address 127.14.16.67:59091)
0807 19:41:11.541599 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:11.541718 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:45053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":288,"threads_started":1}
I20260807 19:41:11.625003 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 29944
I20260807 19:41:11.645272 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 30113
I20260807 19:41:11.663254 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:11.646432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47558 (local address 127.14.16.66:33275)
0807 19:41:11.647002 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:11.647016 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:11.647053 (+ 37us) server_negotiation.cc:407] Connection header received
0807 19:41:11.648666 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:11.648675 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:11.649079 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:11.649359 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:11.650820 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:11.652318 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:11.653569 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:11.654317 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:11.654469 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:11.657282 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:11.657290 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:11.657304 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:11.657342 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:11.660302 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:11.660984 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:11.660991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:11.660997 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:11.661182 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:11.661523 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:11.661526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:11.661529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:11.662035 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:11.662258 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:11.662618 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 19:41:11.662967 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":298,"threads_started":1}
I20260807 19:41:11.664717 30238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:11.748904 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 30113
I20260807 19:41:11.769035 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 30246
I20260807 19:41:11.787694 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:11.770057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40706 (local address 127.14.16.67:44471)
0807 19:41:11.770666 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:11.770681 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:11.770742 (+ 61us) server_negotiation.cc:407] Connection header received
0807 19:41:11.772914 (+ 2172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:11.772924 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:11.773347 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:11.773703 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:11.775210 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:11.776553 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:11.777808 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:11.778470 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:11.778662 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:11.781427 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:11.781435 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:11.781450 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:11.781491 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:11.784334 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:11.785034 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:11.785042 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:11.785055 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:11.785229 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:11.785629 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:11.785635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:11.785639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:11.786163 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:11.786470 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:11.786906 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 19:41:11.787389 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":275,"threads_started":1}
I20260807 19:41:11.789085 30371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:11.872896 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 30246
2026-08-07T19:41:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24520 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:41:11 dist-test-slave-07c2 krb5kdc[30436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:41:11 dist-test-slave-07c2 krb5kdc[30436](info): set up 2 sockets
Aug 07 19:41:11 dist-test-slave-07c2 krb5kdc[30436](info): commencing operation
krb5kdc: starting...
W20260807 19:41:13.013938 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:41:13 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:41:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:41:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:41:13.312760 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:40953
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:40953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:13.656545 30449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:13.656845 30449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:13.657011 30449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:13.664342 30449 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 19:41:13.664441 30449 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 19:41:13.664499 30449 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 19:41:13.664575 30449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:41:13.664661 30449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:13.664778 30449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:13.664858 30449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:41:13.664939 30449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:41:13.674976 30449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:40953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:40953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:41:13.676276 30449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:13.677599 30449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:13.689702 30454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:13.689790 30455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:13.689738 30457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:13.690534 30449 server_base.cc:1034] running on GCE node
I20260807 19:41:13.691342 30449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:13.692759 30449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:13.694005 30449 hybrid_clock.cc:648] HybridClock initialized: now 1786131673693930 us; error 52 us; skew 500 ppm
I20260807 19:41:13.694337 30449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:13 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131673, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:13.697415 30449 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:41:13.698918 30449 webserver.cc:460] Webserver started at http://127.14.16.126:33523/ using document root <none> and password file <none>
I20260807 19:41:13.699674 30449 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:13.699798 30449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:13.700112 30449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:13.703066 30449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "a0836839bd76466ea0d415b805f54402"
format_stamp: "Formatted at 2026-08-07 19:41:13 on dist-test-slave-07c2"
server_key: "c8b191c2feb69587714099efee7fcaa4"
server_key_iv: "60eade3f8790fead05757bc88d53c2f6"
server_key_version: "encryptionkey@0"
I20260807 19:41:13.703907 30449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "a0836839bd76466ea0d415b805f54402"
format_stamp: "Formatted at 2026-08-07 19:41:13 on dist-test-slave-07c2"
server_key: "c8b191c2feb69587714099efee7fcaa4"
server_key_iv: "60eade3f8790fead05757bc88d53c2f6"
server_key_version: "encryptionkey@0"
I20260807 19:41:13.709022 30449 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:41:13.712306 30464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:13.713565 30449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:13.713783 30449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "a0836839bd76466ea0d415b805f54402"
format_stamp: "Formatted at 2026-08-07 19:41:13 on dist-test-slave-07c2"
server_key: "c8b191c2feb69587714099efee7fcaa4"
server_key_iv: "60eade3f8790fead05757bc88d53c2f6"
server_key_version: "encryptionkey@0"
I20260807 19:41:13.714010 30449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:13.726498 30449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:13.730841 30449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:13.731127 30449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:13.750147 30449 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:40953
I20260807 19:41:13.750159 30517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:40953 every 8 connection(s)
I20260807 19:41:13.751684 30449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:41:13.755405 30518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:13.761723 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 30449
I20260807 19:41:13.761952 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:41:13.762606 14401 external_mini_cluster.cc:1428] Setting key e29bbbe8d49cbfad5b6ab3c5c455e08e
I20260807 19:41:13.766378 30518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Bootstrap starting.
I20260807 19:41:13.770308 30518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:13.771492 30518 log.cc:824] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:13.774367 30518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: No bootstrap required, opened a new log
Aug 07 19:41:13 dist-test-slave-07c2 krb5kdc[30436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:41:13.782879 30518 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } }
I20260807 19:41:13.783420 30518 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:13.783558 30518 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0836839bd76466ea0d415b805f54402, State: Initialized, Role: FOLLOWER
I20260807 19:41:13.784232 30518 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } }
I20260807 19:41:13.784497 30518 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:41:13.784662 30518 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:41:13.784853 30518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:13.786947 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:13.764696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33782 (local address 127.14.16.126:40953)
0807 19:41:13.765541 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:13.765557 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:13.765607 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:41:13.768082 (+ 2475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:13.768119 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:13.768686 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:13.769130 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:13.770580 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:13.772059 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:13.773455 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:13.774158 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:13.774379 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:13.779102 (+ 4723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:13.779112 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:13.779130 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:13.779181 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:13.782416 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:13.783332 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:13.783345 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:13.783354 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:13.783652 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:13.784074 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:13.784077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:13.784079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:13.784718 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:13.785013 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:13.785494 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 19:41:13.785954 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":358,"threads_started":1}
I20260807 19:41:13.787508 30518 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } }
I20260807 19:41:13.788046 30518 leader_election.cc:302] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0836839bd76466ea0d415b805f54402; no voters:
I20260807 19:41:13.788652 30518 leader_election.cc:288] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:41:13.788849 30524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:13.789525 30524 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 1 LEADER]: Becoming Leader. State: Replica: a0836839bd76466ea0d415b805f54402, State: Running, Role: LEADER
I20260807 19:41:13.790756 30524 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } }
I20260807 19:41:13.791477 30518 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:41:13.794061 30525 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0836839bd76466ea0d415b805f54402. Latest consensus state: current_term: 1 leader_uuid: "a0836839bd76466ea0d415b805f54402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } } }
I20260807 19:41:13.794516 30525 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:13.794814 30526 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0836839bd76466ea0d415b805f54402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } } }
I20260807 19:41:13.795271 30526 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:13.797480 30533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:41:13.802862 30533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:41:13.812879 30533 catalog_manager.cc:1347] Generated new cluster ID: ecbff516ce8e4e208780f4b1af772a6b
I20260807 19:41:13.813005 30533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:41:13.830969 30533 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:41:13.832361 30533 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:41:13.840394 30533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Generated new TSK 0
I20260807 19:41:13.841194 30533 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:41:13.922770 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40953
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:14.297958 30545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:14.298242 30545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:14.298338 30545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:14.305886 30545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:14.306005 30545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:14.306123 30545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:41:14.316089 30545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:41:14.317400 30545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:14.318787 30545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:14.335691 30550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:14.335651 30551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:14.335762 30553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:14.336740 30545 server_base.cc:1034] running on GCE node
I20260807 19:41:14.337376 30545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:14.338356 30545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:14.339572 30545 hybrid_clock.cc:648] HybridClock initialized: now 1786131674339507 us; error 64 us; skew 500 ppm
I20260807 19:41:14.339919 30545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:14 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131674, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:14.343251 30545 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:41:14.344805 30545 webserver.cc:460] Webserver started at http://127.14.16.65:40677/ using document root <none> and password file <none>
I20260807 19:41:14.345494 30545 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:14.345642 30545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:14.345963 30545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:14.348912 30545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe"
format_stamp: "Formatted at 2026-08-07 19:41:14 on dist-test-slave-07c2"
server_key: "04290f5f6bf8c4e37982f98d3f4abca9"
server_key_iv: "afd7c31d13746a67d2a7a320a6ece99a"
server_key_version: "encryptionkey@0"
I20260807 19:41:14.349664 30545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe"
format_stamp: "Formatted at 2026-08-07 19:41:14 on dist-test-slave-07c2"
server_key: "04290f5f6bf8c4e37982f98d3f4abca9"
server_key_iv: "afd7c31d13746a67d2a7a320a6ece99a"
server_key_version: "encryptionkey@0"
I20260807 19:41:14.354637 30545 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:41:14.358088 30560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:14.359444 30545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:14.359611 30545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe"
format_stamp: "Formatted at 2026-08-07 19:41:14 on dist-test-slave-07c2"
server_key: "04290f5f6bf8c4e37982f98d3f4abca9"
server_key_iv: "afd7c31d13746a67d2a7a320a6ece99a"
server_key_version: "encryptionkey@0"
I20260807 19:41:14.359808 30545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:14.378038 30545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:14.382382 30545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:14.382695 30545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:14.383710 30545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:41:14.385095 30545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:14.385200 30545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:14.385360 30545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:14.385435 30545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:14.419981 30545 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:36845
I20260807 19:41:14.420029 30675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:36845 every 8 connection(s)
I20260807 19:41:14.421250 30545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:41:14.428547 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 30545
I20260807 19:41:14.428776 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:41:14.429445 14401 external_mini_cluster.cc:1428] Setting key 2e03257541d2eec953a8d3a715609683
Aug 07 19:41:14 dist-test-slave-07c2 krb5kdc[30436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131674, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:41:14.441434 30679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:14.423867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:38303 (local address 127.14.16.126:40953)
0807 19:41:14.424329 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:14.424336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:14.425276 (+ 940us) server_negotiation.cc:407] Connection header received
0807 19:41:14.427048 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:14.427053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:14.427127 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:14.427231 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:14.428858 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:14.429685 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:14.431254 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:14.431721 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:14.431880 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:14.435414 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:14.435418 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:14.435422 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:14.435455 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:14.438114 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:14.438910 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:14.438916 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:14.438921 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:14.439061 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:14.439576 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:14.439583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:14.439588 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:14.439881 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:14.440066 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:14.440793 (+ 727us) server_negotiation.cc:299] Negotiation successful
0807 19:41:14.441048 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":210,"threads_started":1}
I20260807 19:41:14.442456 30680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:14.424434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:38303)
0807 19:41:14.425122 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:14.425158 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:14.426795 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:14.427492 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:14.427501 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:14.428042 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:14.428647 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:14.428672 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:14.429852 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:14.429865 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:14.431087 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:14.431100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:14.431970 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:14.431979 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:14.432233 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:14.432939 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:14.432968 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:14.435143 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:14.438278 (+ 3135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:14.438287 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:14.438305 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:14.438769 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:14.439177 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:14.439182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:14.439186 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:14.439454 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:14.440027 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:14.440035 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:14.440404 (+ 369us) client_negotiation.cc:769] Sending connection context
0807 19:41:14.440692 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 19:41:14.441246 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":276,"threads_started":1}
I20260807 19:41:14.444147 30677 heartbeater.cc:347] Connected to a master server at 127.14.16.126:40953
I20260807 19:41:14.444460 30677 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:14.445194 30677 heartbeater.cc:511] Master 127.14.16.126:40953 requested a full tablet report, sending...
I20260807 19:41:14.447554 30483 ts_manager.cc:194] Registered new tserver with Master: a58dc65b6dca44cd87b7fcb9a1b5bbbe (127.14.16.65:36845)
I20260807 19:41:14.449769 30483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:38303
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:41:14.495260 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40953
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:14.830700 30683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:14.830981 30683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:14.831110 30683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:14.838384 30683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:14.838515 30683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:14.838714 30683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:41:14.848654 30683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:41:14.849941 30683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:14.851269 30683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:14.864559 30689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:14.864558 30691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:14.864601 30688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:14.865696 30683 server_base.cc:1034] running on GCE node
I20260807 19:41:14.866292 30683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:14.867254 30683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:14.868583 30683 hybrid_clock.cc:648] HybridClock initialized: now 1786131674868528 us; error 21 us; skew 500 ppm
I20260807 19:41:14.868950 30683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:14 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131674, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:14.872131 30683 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:41:14.873677 30683 webserver.cc:460] Webserver started at http://127.14.16.66:36489/ using document root <none> and password file <none>
I20260807 19:41:14.874387 30683 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:14.874536 30683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:14.874867 30683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:14.877887 30683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "0be50dc8a3934a58a4ab92f1bef80f62"
format_stamp: "Formatted at 2026-08-07 19:41:14 on dist-test-slave-07c2"
server_key: "0c6727f6372c2b9b1bd7933643c5d45e"
server_key_iv: "78eceff018d1fa7bebc17c2de9839fdd"
server_key_version: "encryptionkey@0"
I20260807 19:41:14.878669 30683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "0be50dc8a3934a58a4ab92f1bef80f62"
format_stamp: "Formatted at 2026-08-07 19:41:14 on dist-test-slave-07c2"
server_key: "0c6727f6372c2b9b1bd7933643c5d45e"
server_key_iv: "78eceff018d1fa7bebc17c2de9839fdd"
server_key_version: "encryptionkey@0"
I20260807 19:41:14.883992 30683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:41:14.887470 30698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:14.888747 30683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:41:14.888927 30683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "0be50dc8a3934a58a4ab92f1bef80f62"
format_stamp: "Formatted at 2026-08-07 19:41:14 on dist-test-slave-07c2"
server_key: "0c6727f6372c2b9b1bd7933643c5d45e"
server_key_iv: "78eceff018d1fa7bebc17c2de9839fdd"
server_key_version: "encryptionkey@0"
I20260807 19:41:14.889140 30683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:14.909211 30683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:14.913762 30683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:14.914067 30683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:14.915082 30683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:41:14.916488 30683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:14.916594 30683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:14.916739 30683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:14.916810 30683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:14.950640 30683 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:40761
I20260807 19:41:14.950878 30812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:40761 every 8 connection(s)
I20260807 19:41:14.951996 30683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:41:14.955340 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 30683
I20260807 19:41:14.955547 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:41:14.956025 14401 external_mini_cluster.cc:1428] Setting key 264d0ddc1d0601b131fdb91c69effe74
Aug 07 19:41:14 dist-test-slave-07c2 krb5kdc[30436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131674, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:41:14.974112 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:14.954588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:48041 (local address 127.14.16.126:40953)
0807 19:41:14.955129 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:14.955137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:14.956180 (+ 1043us) server_negotiation.cc:407] Connection header received
0807 19:41:14.958515 (+ 2335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:14.958522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:14.958601 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:14.958762 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:14.960219 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:14.961376 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:14.962870 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:14.963665 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:14.963822 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:14.967395 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:14.967404 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:14.967412 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:14.967458 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:14.970867 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:14.971845 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:14.971848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:14.971851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:14.971977 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:14.972458 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:14.972461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:14.972463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:14.972715 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:14.972833 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:14.973654 (+ 821us) server_negotiation.cc:299] Negotiation successful
0807 19:41:14.973833 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":225,"threads_started":1}
I20260807 19:41:14.975625 30816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:14.955131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:48041)
0807 19:41:14.955987 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:14.956040 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:14.958258 (+ 2218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:14.958927 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:14.958935 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:14.959511 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:14.960046 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:14.960062 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:14.961523 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:14.961526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:14.962720 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:14.962731 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:14.963875 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:14.963880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:14.964065 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:14.964796 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:14.964817 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:14.967140 (+ 2323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:14.971005 (+ 3865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:14.971016 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:14.971032 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:14.971729 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:14.972090 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:14.972097 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:14.972104 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:14.972348 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:14.972855 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:14.972865 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:14.973291 (+ 426us) client_negotiation.cc:769] Sending connection context
0807 19:41:14.973579 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 19:41:14.974140 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":399,"threads_started":1}
I20260807 19:41:14.977123 30813 heartbeater.cc:347] Connected to a master server at 127.14.16.126:40953
I20260807 19:41:14.977432 30813 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:14.978137 30813 heartbeater.cc:511] Master 127.14.16.126:40953 requested a full tablet report, sending...
I20260807 19:41:14.979652 30483 ts_manager.cc:194] Registered new tserver with Master: 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761)
I20260807 19:41:14.980947 30483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:48041
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:41:15.023802 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40953
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:15.358349 30819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:15.358628 30819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:15.358724 30819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:15.365923 30819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:15.366024 30819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:15.366132 30819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:41:15.375458 30819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:41:15.376587 30819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:15.377827 30819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:15.390782 30824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:15.390758 30825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:15.390923 30827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:15.391691 30819 server_base.cc:1034] running on GCE node
I20260807 19:41:15.392431 30819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:15.393497 30819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:15.394762 30819 hybrid_clock.cc:648] HybridClock initialized: now 1786131675394705 us; error 65 us; skew 500 ppm
I20260807 19:41:15.395077 30819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:15 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131675, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:15.398407 30819 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:41:15.400087 30819 webserver.cc:460] Webserver started at http://127.14.16.67:34115/ using document root <none> and password file <none>
I20260807 19:41:15.400826 30819 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:15.400987 30819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:15.401309 30819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:15.404330 30819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "d62d30da868c4d5d9e4e398f5601d6b3"
format_stamp: "Formatted at 2026-08-07 19:41:15 on dist-test-slave-07c2"
server_key: "a40fbc738e7a836b05b7203771fbdb0c"
server_key_iv: "9f2db8e7cc4d5071c1f89e7fc8ae65fd"
server_key_version: "encryptionkey@0"
I20260807 19:41:15.405097 30819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "d62d30da868c4d5d9e4e398f5601d6b3"
format_stamp: "Formatted at 2026-08-07 19:41:15 on dist-test-slave-07c2"
server_key: "a40fbc738e7a836b05b7203771fbdb0c"
server_key_iv: "9f2db8e7cc4d5071c1f89e7fc8ae65fd"
server_key_version: "encryptionkey@0"
I20260807 19:41:15.410247 30819 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 19:41:15.413664 30834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:15.415025 30819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:41:15.415223 30819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "d62d30da868c4d5d9e4e398f5601d6b3"
format_stamp: "Formatted at 2026-08-07 19:41:15 on dist-test-slave-07c2"
server_key: "a40fbc738e7a836b05b7203771fbdb0c"
server_key_iv: "9f2db8e7cc4d5071c1f89e7fc8ae65fd"
server_key_version: "encryptionkey@0"
I20260807 19:41:15.415489 30819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:15.438861 30819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:15.443092 30819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:15.443466 30819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:15.444466 30819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:41:15.445796 30819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:15.445904 30819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:15.446056 30819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:15.446128 30819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:15.452857 30677 heartbeater.cc:503] Master 127.14.16.126:40953 was elected leader, sending a full tablet report...
I20260807 19:41:15.480233 30819 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:42833
I20260807 19:41:15.480321 30948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:42833 every 8 connection(s)
I20260807 19:41:15.481531 30819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:41:15.482556 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 30819
I20260807 19:41:15.482789 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:41:15.483387 14401 external_mini_cluster.cc:1428] Setting key 8e259659a450a9412f9d0a1d5bd1f126
Aug 07 19:41:15 dist-test-slave-07c2 krb5kdc[30436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131675, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:41:15.502961 30951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.484554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:48127 (local address 127.14.16.126:40953)
0807 19:41:15.485094 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:15.485098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:15.485973 (+ 875us) server_negotiation.cc:407] Connection header received
0807 19:41:15.488347 (+ 2374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:15.488355 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:15.488464 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:15.488595 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:15.490118 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.490927 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.492768 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.493326 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.493460 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.497002 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:15.497007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:15.497010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:15.497042 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:15.500247 (+ 3205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:15.500917 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:15.500920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:15.500923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:15.501037 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:15.501469 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:15.501472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:15.501474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:15.501744 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:15.501870 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:15.502513 (+ 643us) server_negotiation.cc:299] Negotiation successful
0807 19:41:15.502676 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":280,"threads_started":1}
I20260807 19:41:15.503981 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.484868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:48127)
0807 19:41:15.485810 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:15.485867 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:15.488066 (+ 2199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:15.488844 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:15.488852 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:15.489375 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:15.489938 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:15.489956 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.491087 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.491093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:15.492604 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:15.492617 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.493575 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.493580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:15.493783 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.494410 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:15.494450 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:15.496810 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:15.500370 (+ 3560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:15.500376 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:15.500388 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:15.500809 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:15.501150 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:15.501153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:15.501156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:15.501356 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:15.501854 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:15.501860 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:15.502170 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 19:41:15.502411 (+ 241us) client_negotiation.cc:240] Negotiation successful
0807 19:41:15.502794 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":369,"threads_started":1}
I20260807 19:41:15.505242 30949 heartbeater.cc:347] Connected to a master server at 127.14.16.126:40953
I20260807 19:41:15.505546 30949 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:15.506237 30949 heartbeater.cc:511] Master 127.14.16.126:40953 requested a full tablet report, sending...
I20260807 19:41:15.507683 30483 ts_manager.cc:194] Registered new tserver with Master: d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833)
I20260807 19:41:15.508438 30483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:48127
I20260807 19:41:15.511821 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:41:15.528632 30951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.516297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33790 (local address 127.14.16.126:40953)
0807 19:41:15.516450 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:15.516454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:15.516941 (+ 487us) server_negotiation.cc:407] Connection header received
0807 19:41:15.517076 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:15.517079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:15.517138 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:15.517219 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:15.518754 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.519583 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.521082 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.521468 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.521559 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.523845 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:15.523849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:15.523852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:15.523879 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:15.526208 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:15.526817 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:15.526821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:15.526823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:15.526934 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:15.527265 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:15.527268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:15.527270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:15.527628 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:15.527816 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:15.528158 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 19:41:15.528376 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:41:15.532639 30483 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:41:15.539626 30483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:41:15.563786 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.554285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:40761 (local address 127.0.0.1:36990)
0807 19:41:15.554966 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:15.554982 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:15.555109 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:15.556331 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:15.556338 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:15.556377 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:15.556850 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:15.556869 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.559094 (+ 2225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.559099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:15.561333 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:15.561350 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.562880 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.562884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:15.563015 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.563101 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 19:41:15.563216 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 19:41:15.563417 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":3,"thread_start_us":573,"threads_started":1}
I20260807 19:41:15.563820 30968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.554714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36990 (local address 127.14.16.66:40761)
0807 19:41:15.555646 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:15.555657 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:15.555690 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:41:15.555812 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:15.555820 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:15.556087 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:15.557084 (+ 997us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:15.557313 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.558943 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.561534 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.562699 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.562850 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.563112 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:15.563331 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 19:41:15.563506 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":689,"threads_started":1}
I20260807 19:41:15.566376 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.552027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:42833 (local address 127.0.0.1:60496)
0807 19:41:15.553010 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:15.553073 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:15.553291 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:15.557187 (+ 3896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:15.557191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:15.557223 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:15.557733 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:15.557747 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.559459 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.559466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:15.562088 (+ 2622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:15.562103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.565046 (+ 2943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.565053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:15.565585 (+ 532us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.565701 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 19:41:15.565795 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 19:41:15.565972 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":548,"thread_start_us":686,"threads_started":1}
I20260807 19:41:15.566370 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.552264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.14.16.67:42833)
0807 19:41:15.556578 (+ 4314us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:15.556586 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:15.556622 (+ 36us) server_negotiation.cc:407] Connection header received
0807 19:41:15.556716 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:15.556733 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:15.556963 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:15.557934 (+ 971us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:15.558165 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.559254 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.562262 (+ 3008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.564770 (+ 2508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.565204 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.565446 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:15.565843 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 19:41:15.566013 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4121,"thread_start_us":3954,"threads_started":1}
I20260807 19:41:15.566690 30966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.552535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.14.16.65:36845)
0807 19:41:15.556093 (+ 3558us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:15.556101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:15.556129 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:41:15.556238 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:15.556248 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:15.556462 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:15.557447 (+ 985us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:15.557692 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.559166 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.563182 (+ 4016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.564786 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.565907 (+ 1121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.566125 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:15.566192 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 19:41:15.566361 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3389,"thread_start_us":3230,"threads_started":1}
I20260807 19:41:15.567274 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.552092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:36845 (local address 127.0.0.1:43374)
0807 19:41:15.553925 (+ 1833us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:15.553945 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:15.554068 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:15.556685 (+ 2617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:15.556690 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:15.556714 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:15.557176 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:15.557191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.559330 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.559337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:15.562089 (+ 2752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:15.562103 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.565059 (+ 2956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.565065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:15.565583 (+ 518us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.565701 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 19:41:15.565790 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 19:41:15.566997 (+ 1207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1683,"mutex_wait_us":1218,"thread_start_us":2271,"threads_started":1}
I20260807 19:41:15.568305 30747 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0efe29754840ca84c33e6edb9b5e48 (DEFAULT_TABLE table=test-table [id=4a83bae210d9490e8146f91ffbba459b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:15.569904 30747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0efe29754840ca84c33e6edb9b5e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:15.569831 30883 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0efe29754840ca84c33e6edb9b5e48 (DEFAULT_TABLE table=test-table [id=4a83bae210d9490e8146f91ffbba459b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:15.571251 30883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0efe29754840ca84c33e6edb9b5e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:15.570775 30609 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0efe29754840ca84c33e6edb9b5e48 (DEFAULT_TABLE table=test-table [id=4a83bae210d9490e8146f91ffbba459b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:15.572113 30609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0efe29754840ca84c33e6edb9b5e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:15.584574 30969 tablet_bootstrap.cc:492] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62: Bootstrap starting.
I20260807 19:41:15.588050 30970 tablet_bootstrap.cc:492] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3: Bootstrap starting.
I20260807 19:41:15.588771 30969 tablet_bootstrap.cc:654] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:15.590111 30969 log.cc:824] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:15.590991 30971 tablet_bootstrap.cc:492] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Bootstrap starting.
I20260807 19:41:15.591830 30970 tablet_bootstrap.cc:654] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:15.593134 30970 log.cc:824] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:15.594549 30971 tablet_bootstrap.cc:654] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:15.594913 30969 tablet_bootstrap.cc:492] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62: No bootstrap required, opened a new log
I20260807 19:41:15.595257 30969 ts_tablet_manager.cc:1397] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 19:41:15.595841 30971 log.cc:824] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:15.596475 30970 tablet_bootstrap.cc:492] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3: No bootstrap required, opened a new log
I20260807 19:41:15.596838 30970 ts_tablet_manager.cc:1397] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 19:41:15.598400 30971 tablet_bootstrap.cc:492] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: No bootstrap required, opened a new log
I20260807 19:41:15.598665 30971 ts_tablet_manager.cc:1397] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:41:15.605803 30971 raft_consensus.cc:348] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:15.606637 30971 raft_consensus.cc:374] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:15.606389 30969 raft_consensus.cc:348] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:15.606806 30971 raft_consensus.cc:729] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58dc65b6dca44cd87b7fcb9a1b5bbbe, State: Initialized, Role: FOLLOWER
I20260807 19:41:15.606942 30969 raft_consensus.cc:374] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:15.607100 30969 raft_consensus.cc:729] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be50dc8a3934a58a4ab92f1bef80f62, State: Initialized, Role: FOLLOWER
I20260807 19:41:15.607699 30971 consensus_queue.cc:260] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:15.607677 30970 raft_consensus.cc:348] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:15.607905 30969 consensus_queue.cc:260] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:15.608134 30970 raft_consensus.cc:374] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:15.608223 30970 raft_consensus.cc:729] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62d30da868c4d5d9e4e398f5601d6b3, State: Initialized, Role: FOLLOWER
I20260807 19:41:15.609124 30970 consensus_queue.cc:260] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:15.609719 30971 ts_tablet_manager.cc:1428] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260807 19:41:15.610742 30949 heartbeater.cc:503] Master 127.14.16.126:40953 was elected leader, sending a full tablet report...
I20260807 19:41:15.611646 30970 ts_tablet_manager.cc:1428] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260807 19:41:15.612607 30813 heartbeater.cc:503] Master 127.14.16.126:40953 was elected leader, sending a full tablet report...
I20260807 19:41:15.613296 30969 ts_tablet_manager.cc:1428] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62: Time spent starting tablet: real 0.018s user 0.016s sys 0.000s
W20260807 19:41:15.673856 30678 tablet.cc:2369] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:41:15.704063 30814 tablet.cc:2369] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:41:15.733189 30950 tablet.cc:2369] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:41:15.748741 30978 raft_consensus.cc:482] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:41:15.749028 30978 raft_consensus.cc:504] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:15.751013 30978 leader_election.cc:288] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a58dc65b6dca44cd87b7fcb9a1b5bbbe (127.14.16.65:36845), 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761)
I20260807 19:41:15.757643 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.751238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:40761 (local address 127.14.16.67:49787)
0807 19:41:15.751559 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:15.751582 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:15.751723 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:15.752237 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:15.752243 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:15.752281 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:15.752490 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:15.752501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.753855 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.753861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:15.755358 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:15.755375 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.756907 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.756913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:15.757096 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.757131 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:41:15.757230 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 19:41:15.757368 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260807 19:41:15.757674 30968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.751611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:49787 (local address 127.14.16.66:40761)
0807 19:41:15.751799 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:15.751804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:15.751823 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:41:15.751894 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:15.751899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:15.751969 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:15.752060 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:15.752658 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.753717 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.755527 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.756741 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.756905 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.756979 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:15.757284 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 19:41:15.757425 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:41:15.758795 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.751243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:36845 (local address 127.14.16.67:51533)
0807 19:41:15.752591 (+ 1348us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:15.752610 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:15.752751 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:15.753085 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:15.753090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:15.753115 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:15.753452 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:15.753462 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.754848 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.754853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:15.756365 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:15.756376 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.758046 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.758052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:15.758183 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.758212 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 19:41:15.758311 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 19:41:15.758427 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1224,"mutex_wait_us":591,"thread_start_us":360,"threads_started":1}
I20260807 19:41:15.758917 30767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0efe29754840ca84c33e6edb9b5e48" candidate_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" is_pre_election: true
I20260807 19:41:15.759502 30966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.751632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:51533 (local address 127.14.16.65:36845)
0807 19:41:15.751839 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:15.751845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:15.752723 (+ 878us) server_negotiation.cc:407] Connection header received
0807 19:41:15.752881 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:15.752886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:15.752957 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:15.753042 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:15.753617 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.754676 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.756532 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.757657 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.757831 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.757907 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:15.758353 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 19:41:15.759253 (+ 900us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:41:15.759541 30767 raft_consensus.cc:2454] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62d30da868c4d5d9e4e398f5601d6b3 in term 0.
I20260807 19:41:15.760325 30836 leader_election.cc:302] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0be50dc8a3934a58a4ab92f1bef80f62, d62d30da868c4d5d9e4e398f5601d6b3; no voters:
I20260807 19:41:15.760434 30630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0efe29754840ca84c33e6edb9b5e48" candidate_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" is_pre_election: true
I20260807 19:41:15.760754 30978 raft_consensus.cc:2781] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:41:15.760900 30978 raft_consensus.cc:482] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:41:15.760987 30978 raft_consensus.cc:3037] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:15.760994 30630 raft_consensus.cc:2454] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62d30da868c4d5d9e4e398f5601d6b3 in term 0.
I20260807 19:41:15.764230 30978 raft_consensus.cc:504] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:15.764906 30978 leader_election.cc:288] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [CANDIDATE]: Term 1 election: Requested vote from peers a58dc65b6dca44cd87b7fcb9a1b5bbbe (127.14.16.65:36845), 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761)
I20260807 19:41:15.765309 30630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0efe29754840ca84c33e6edb9b5e48" candidate_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe"
I20260807 19:41:15.765348 30767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0efe29754840ca84c33e6edb9b5e48" candidate_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be50dc8a3934a58a4ab92f1bef80f62"
I20260807 19:41:15.765511 30630 raft_consensus.cc:3037] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:15.765547 30767 raft_consensus.cc:3037] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:15.767809 30767 raft_consensus.cc:2454] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62d30da868c4d5d9e4e398f5601d6b3 in term 1.
I20260807 19:41:15.768220 30836 leader_election.cc:302] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0be50dc8a3934a58a4ab92f1bef80f62, d62d30da868c4d5d9e4e398f5601d6b3; no voters:
I20260807 19:41:15.768508 30978 raft_consensus.cc:2781] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:15.768656 30630 raft_consensus.cc:2454] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62d30da868c4d5d9e4e398f5601d6b3 in term 1.
I20260807 19:41:15.769960 30978 raft_consensus.cc:686] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 1 LEADER]: Becoming Leader. State: Replica: d62d30da868c4d5d9e4e398f5601d6b3, State: Running, Role: LEADER
I20260807 19:41:15.770438 30978 consensus_queue.cc:237] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:15.775007 30481 catalog_manager.cc:5495] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67). New cstate: current_term: 1 leader_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } health_report { overall_health: UNKNOWN } } }
I20260807 19:41:15.805486 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.800292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60512 (local address 127.14.16.67:42833)
0807 19:41:15.800506 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:15.800517 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:15.800535 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:41:15.800639 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:15.800643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:15.800727 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:15.800846 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:15.801175 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.801877 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.803449 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.803843 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.803938 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.804415 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:15.804910 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:41:15.805019 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:15.805107 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 19:41:15.805172 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:41:15.813903 30767 raft_consensus.cc:1261] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 1 FOLLOWER]: Refusing update from remote peer d62d30da868c4d5d9e4e398f5601d6b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:41:15.813903 30630 raft_consensus.cc:1261] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Refusing update from remote peer d62d30da868c4d5d9e4e398f5601d6b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:41:15.815006 30980 consensus_queue.cc:1035] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:15.815474 30978 consensus_queue.cc:1035] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:15.844800 30483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:41:15.845077 30483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:41:15.848788 30483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Sending DeleteTablet for 3 replicas of tablet 5b0efe29754840ca84c33e6edb9b5e48
I20260807 19:41:15.849936 30883 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b0efe29754840ca84c33e6edb9b5e48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:15 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:60496
I20260807 19:41:15.850023 30747 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b0efe29754840ca84c33e6edb9b5e48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:15 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:36990
I20260807 19:41:15.850041 30609 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b0efe29754840ca84c33e6edb9b5e48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:15 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:43374
I20260807 19:41:15.852319 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 30545
I20260807 19:41:15.860795 30995 tablet_replica.cc:330] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: stopping tablet replica
I20260807 19:41:15.861420 30995 raft_consensus.cc:2229] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:41:15.862099 30995 raft_consensus.cc:2258] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:15.862710 30994 tablet_replica.cc:330] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3: stopping tablet replica
I20260807 19:41:15.863669 30993 tablet_replica.cc:330] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62: stopping tablet replica
I20260807 19:41:15.864197 30994 raft_consensus.cc:2229] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:41:15.864759 30993 raft_consensus.cc:2229] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:41:15.865326 30993 raft_consensus.cc:2258] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:15.865924 30995 ts_tablet_manager.cc:1905] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:15.866014 30994 raft_consensus.cc:2258] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:15.868444 30993 ts_tablet_manager.cc:1905] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:15.870429 30994 ts_tablet_manager.cc:1905] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:15.872733 30995 ts_tablet_manager.cc:1918] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:41:15.872905 30995 log.cc:1196] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/5b0efe29754840ca84c33e6edb9b5e48
I20260807 19:41:15.873551 30995 ts_tablet_manager.cc:1939] T 5b0efe29754840ca84c33e6edb9b5e48 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Deleting consensus metadata
I20260807 19:41:15.875587 30465 catalog_manager.cc:4841] TS a58dc65b6dca44cd87b7fcb9a1b5bbbe (127.14.16.65:36845): tablet 5b0efe29754840ca84c33e6edb9b5e48 (table test-table [id=4a83bae210d9490e8146f91ffbba459b]) successfully deleted
I20260807 19:41:15.879555 30993 ts_tablet_manager.cc:1918] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 07 19:41:15 dist-test-slave-07c2 krb5kdc[30436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:41:15.879841 30993 log.cc:1196] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/5b0efe29754840ca84c33e6edb9b5e48
I20260807 19:41:15.880483 30993 ts_tablet_manager.cc:1939] T 5b0efe29754840ca84c33e6edb9b5e48 P 0be50dc8a3934a58a4ab92f1bef80f62: Deleting consensus metadata
I20260807 19:41:15.880466 30994 ts_tablet_manager.cc:1918] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:41:15.880640 30994 log.cc:1196] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/5b0efe29754840ca84c33e6edb9b5e48
I20260807 19:41:15.881253 30994 ts_tablet_manager.cc:1939] T 5b0efe29754840ca84c33e6edb9b5e48 P d62d30da868c4d5d9e4e398f5601d6b3: Deleting consensus metadata
I20260807 19:41:15.882536 30466 catalog_manager.cc:4841] TS 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761): tablet 5b0efe29754840ca84c33e6edb9b5e48 (table test-table [id=4a83bae210d9490e8146f91ffbba459b]) successfully deleted
I20260807 19:41:15.883014 30467 catalog_manager.cc:4841] TS d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833): tablet 5b0efe29754840ca84c33e6edb9b5e48 (table test-table [id=4a83bae210d9490e8146f91ffbba459b]) successfully deleted
I20260807 19:41:15.887465 30966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:15.853827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43380 (local address 127.14.16.65:36845)
0807 19:41:15.854062 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:15.854070 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:15.863748 (+ 9678us) server_negotiation.cc:407] Connection header received
0807 19:41:15.864128 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:15.864133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:15.864215 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:15.864292 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:15.866508 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.867672 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.869748 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:15.870362 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:15.870462 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:15.881144 (+ 10682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:15.881161 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:15.881179 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:15.881242 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:15.884522 (+ 3280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:15.885258 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:15.885271 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:15.885273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:15.885404 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:15.885830 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:15.885836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:15.885839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:15.886196 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:15.886516 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:15.886827 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 19:41:15.887070 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:41:15.888167 30671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:15.980674 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 30545
I20260807 19:41:16.002594 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 30683
Aug 07 19:41:16 dist-test-slave-07c2 krb5kdc[30436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:41:16.020735 30968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:16.003552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37006 (local address 127.14.16.66:40761)
0807 19:41:16.003782 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:16.003791 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:16.003814 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:41:16.003885 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:16.003890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:16.003975 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:16.004087 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:16.005486 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:16.006687 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:16.008194 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:16.008880 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:16.009012 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:16.013503 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:16.013524 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:16.013544 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:16.013598 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:16.017545 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:16.018305 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:16.018314 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:16.018318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:16.018479 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:16.018928 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:16.018932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:16.018935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:16.019470 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:16.019814 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:16.020106 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 19:41:16.020333 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:41:16.021497 30808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:16.113405 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 30683
I20260807 19:41:16.139793 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 30819
Aug 07 19:41:16 dist-test-slave-07c2 krb5kdc[30436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:41:16.158934 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:16.140676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60526 (local address 127.14.16.67:42833)
0807 19:41:16.140854 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:16.140858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:16.140929 (+ 71us) server_negotiation.cc:407] Connection header received
0807 19:41:16.141130 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:16.141134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:16.141226 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:16.141314 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:16.143064 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:16.143936 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:16.145873 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:16.146375 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:16.146508 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:16.152276 (+ 5768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:16.152291 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:16.152306 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:16.152351 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:16.155779 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:16.156596 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:16.156607 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:16.156610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:16.156788 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:16.157213 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:16.157219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:16.157232 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:16.157610 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:16.157948 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:16.158283 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 19:41:16.158522 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:41:16.159821 30944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:16.285893 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 30819
I20260807 19:41:16.308497 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 30449
I20260807 19:41:16.309732 30513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:16.421244 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 30449
I20260807 19:41:36.446010 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:40953
--webserver_interface=127.14.16.126
--webserver_port=33523
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:40953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:36.789954 31001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:36.790226 31001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:36.790402 31001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:36.797480 31001 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 19:41:36.797572 31001 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 19:41:36.797653 31001 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 19:41:36.797770 31001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:41:36.797863 31001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:36.797987 31001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:36.798067 31001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:41:36.798151 31001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:41:36.807909 31001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:40953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:40953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=33523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:41:36.809214 31001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:36.810568 31001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:36.836053 31008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:36.836086 31007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:36.836133 31010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:36.837335 31001 server_base.cc:1034] running on GCE node
I20260807 19:41:36.838243 31001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:36.839906 31001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:36.841163 31001 hybrid_clock.cc:648] HybridClock initialized: now 1786131696841097 us; error 42 us; skew 500 ppm
I20260807 19:41:36.841583 31001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:36 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131696, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:36.849061 31001 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:41:36.851006 31001 webserver.cc:460] Webserver started at http://127.14.16.126:33523/ using document root <none> and password file <none>
I20260807 19:41:36.851908 31001 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:36.852066 31001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:36.858206 31001 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 19:41:36.861240 31017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:36.862561 31001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:41:36.862757 31001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "a0836839bd76466ea0d415b805f54402"
format_stamp: "Formatted at 2026-08-07 19:41:13 on dist-test-slave-07c2"
server_key: "c8b191c2feb69587714099efee7fcaa4"
server_key_iv: "60eade3f8790fead05757bc88d53c2f6"
server_key_version: "encryptionkey@0"
I20260807 19:41:36.863929 31001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:36.879369 31001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:36.883594 31001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:36.883915 31001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:36.903379 31001 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:40953
I20260807 19:41:36.903419 31070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:40953 every 8 connection(s)
I20260807 19:41:36.904989 31001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:41:36.906813 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 31001
I20260807 19:41:36.907742 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:36845
--local_ip_for_outbound_sockets=127.14.16.65
--tserver_master_addrs=127.14.16.126:40953
--webserver_port=40677
--webserver_interface=127.14.16.65
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:41:36.912286 31071 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:41:36.916266 31071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Bootstrap starting.
I20260807 19:41:36.936923 31071 log.cc:824] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:36.949398 31071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:41:36.949966 31071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Bootstrap complete.
I20260807 19:41:36.959525 31071 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } }
I20260807 19:41:36.960453 31071 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0836839bd76466ea0d415b805f54402, State: Initialized, Role: FOLLOWER
I20260807 19:41:36.961223 31071 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } }
I20260807 19:41:36.961514 31071 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:41:36.961663 31071 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:41:36.961858 31071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:41:36.965915 31071 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } }
I20260807 19:41:36.966463 31071 leader_election.cc:302] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0836839bd76466ea0d415b805f54402; no voters:
I20260807 19:41:36.967145 31071 leader_election.cc:288] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:41:36.967411 31075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:41:36.968549 31075 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 2 LEADER]: Becoming Leader. State: Replica: a0836839bd76466ea0d415b805f54402, State: Running, Role: LEADER
I20260807 19:41:36.969236 31075 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } }
I20260807 19:41:36.969677 31071 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:41:36.971827 31078 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0836839bd76466ea0d415b805f54402. Latest consensus state: current_term: 2 leader_uuid: "a0836839bd76466ea0d415b805f54402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } } }
I20260807 19:41:36.972581 31078 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:36.972451 31076 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a0836839bd76466ea0d415b805f54402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } } }
I20260807 19:41:36.973610 31076 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:36.976105 31084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:41:36.985461 31084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:41:36.987005 31084 catalog_manager.cc:1259] Loaded cluster ID: ecbff516ce8e4e208780f4b1af772a6b
I20260807 19:41:36.987169 31084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:41:36.992209 31084 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:41:36.994357 31084 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Loaded TSK: 0
I20260807 19:41:37.028167 31084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Generated new TSK 1
I20260807 19:41:37.028589 31084 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:41:37.269179 31073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:37.269528 31073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:37.269690 31073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:37.277272 31073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:37.277431 31073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:37.277622 31073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:41:37.287698 31073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:36845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=40677
--tserver_master_addrs=127.14.16.126:40953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:41:37.289144 31073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:37.290680 31073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:37.306516 31099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:37.306555 31100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:37.307380 31102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:37.314517 31073 server_base.cc:1034] running on GCE node
I20260807 19:41:37.315454 31073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:37.316798 31073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:37.318066 31073 hybrid_clock.cc:648] HybridClock initialized: now 1786131697318011 us; error 61 us; skew 500 ppm
I20260807 19:41:37.318495 31073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:37 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131697, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:37.322726 31073 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:41:37.324560 31073 webserver.cc:460] Webserver started at http://127.14.16.65:40677/ using document root <none> and password file <none>
I20260807 19:41:37.325486 31073 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:37.325650 31073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:37.332782 31073 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:41:37.335956 31109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:37.337477 31073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:41:37.337636 31073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe"
format_stamp: "Formatted at 2026-08-07 19:41:14 on dist-test-slave-07c2"
server_key: "04290f5f6bf8c4e37982f98d3f4abca9"
server_key_iv: "afd7c31d13746a67d2a7a320a6ece99a"
server_key_version: "encryptionkey@0"
I20260807 19:41:37.338933 31073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:37.362048 31073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:37.367467 31073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:37.367810 31073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:37.369017 31073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:41:37.370705 31073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:37.370814 31073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:37.370913 31073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:37.370993 31073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:37.409071 31073 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:36845
I20260807 19:41:37.409164 31225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:36845 every 8 connection(s)
I20260807 19:41:37.410593 31073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:41:37.419260 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 31073
I20260807 19:41:37.419960 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:40761
--local_ip_for_outbound_sockets=127.14.16.66
--tserver_master_addrs=127.14.16.126:40953
--webserver_port=36489
--webserver_interface=127.14.16.66
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:41:37 dist-test-slave-07c2 krb5kdc[30436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131697, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:41:37.436116 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:37.413728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:39797)
0807 19:41:37.414548 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:37.414594 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:37.416373 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:37.417659 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:37.417666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:37.418171 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:37.418741 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:37.418778 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:37.420609 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:37.420617 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:37.422798 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:37.422818 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:37.423927 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:37.423936 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:37.424258 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:37.425021 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:37.425056 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:37.427901 (+ 2845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:37.431537 (+ 3636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:37.431547 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:37.431562 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:37.432015 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:37.432496 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:37.432501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:37.432504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:37.432704 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:37.433503 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:37.433512 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:37.433858 (+ 346us) client_negotiation.cc:769] Sending connection context
0807 19:41:37.434133 (+ 275us) client_negotiation.cc:240] Negotiation successful
0807 19:41:37.434731 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":365,"threads_started":1}
I20260807 19:41:37.436117 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:37.413540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:39797 (local address 127.14.16.126:40953)
0807 19:41:37.414306 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:37.414323 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:37.414740 (+ 417us) server_negotiation.cc:407] Connection header received
0807 19:41:37.416899 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:37.416913 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:37.417227 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:37.417531 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:37.418974 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:37.420440 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:37.422945 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:37.423677 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:37.423823 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:37.428218 (+ 4395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:37.428227 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:37.428242 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:37.428287 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:37.431378 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:37.432167 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:37.432183 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:37.432189 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:37.432363 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:37.432829 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:37.432835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:37.432838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:37.433349 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:37.433553 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:37.434354 (+ 801us) server_negotiation.cc:299] Negotiation successful
0807 19:41:37.434749 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":256,"threads_started":1}
I20260807 19:41:37.439466 31226 heartbeater.cc:347] Connected to a master server at 127.14.16.126:40953
I20260807 19:41:37.439798 31226 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:37.440730 31226 heartbeater.cc:511] Master 127.14.16.126:40953 requested a full tablet report, sending...
I20260807 19:41:37.443218 31035 ts_manager.cc:194] Registered new tserver with Master: a58dc65b6dca44cd87b7fcb9a1b5bbbe (127.14.16.65:36845)
I20260807 19:41:37.444943 31035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:39797
W20260807 19:41:37.767812 31230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:37.768175 31230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:37.768352 31230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:37.776034 31230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:37.776160 31230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:37.776275 31230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:41:37.786163 31230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:40761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=36489
--tserver_master_addrs=127.14.16.126:40953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:41:37.787426 31230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:37.789047 31230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:37.802335 31236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:37.802285 31238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:37.802376 31235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:37.803393 31230 server_base.cc:1034] running on GCE node
I20260807 19:41:37.803995 31230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:37.804898 31230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:37.806128 31230 hybrid_clock.cc:648] HybridClock initialized: now 1786131697806080 us; error 59 us; skew 500 ppm
I20260807 19:41:37.806438 31230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:37 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131697, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:37.809463 31230 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:41:37.810917 31230 webserver.cc:460] Webserver started at http://127.14.16.66:36489/ using document root <none> and password file <none>
I20260807 19:41:37.811664 31230 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:37.811854 31230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:37.817131 31230 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 19:41:37.819650 31246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:37.820930 31230 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:37.821105 31230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "0be50dc8a3934a58a4ab92f1bef80f62"
format_stamp: "Formatted at 2026-08-07 19:41:14 on dist-test-slave-07c2"
server_key: "0c6727f6372c2b9b1bd7933643c5d45e"
server_key_iv: "78eceff018d1fa7bebc17c2de9839fdd"
server_key_version: "encryptionkey@0"
I20260807 19:41:37.822090 31230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:37.837466 31230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:37.841334 31230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:37.841555 31230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:37.842473 31230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:41:37.843791 31230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:37.843884 31230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:37.843961 31230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:37.844022 31230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:37.878191 31230 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:40761
I20260807 19:41:37.878280 31363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:40761 every 8 connection(s)
I20260807 19:41:37.879701 31230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:41:37.883426 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 31230
I20260807 19:41:37.884132 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:42833
--local_ip_for_outbound_sockets=127.14.16.67
--tserver_master_addrs=127.14.16.126:40953
--webserver_port=34115
--webserver_interface=127.14.16.67
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:41:37 dist-test-slave-07c2 krb5kdc[30436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131697, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:41:37.902041 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:37.882270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:35565 (local address 127.14.16.126:40953)
0807 19:41:37.882459 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:37.882463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:37.883766 (+ 1303us) server_negotiation.cc:407] Connection header received
0807 19:41:37.885659 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:37.885668 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:37.885771 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:37.885893 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:37.887378 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:37.888522 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:37.890111 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:37.890896 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:37.891042 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:37.895397 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:37.895403 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:37.895406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:37.895442 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:37.898802 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:37.899674 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:37.899679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:37.899681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:37.899808 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:37.900279 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:37.900283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:37.900285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:37.900579 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:37.900751 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:37.901436 (+ 685us) server_negotiation.cc:299] Negotiation successful
0807 19:41:37.901692 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:41:37.903139 31366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:37.882742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:35565)
0807 19:41:37.883572 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:37.883625 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:37.885388 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:37.886084 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:37.886092 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:37.886575 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:37.887189 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:37.887206 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:37.888738 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:37.888742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:37.889928 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:37.889940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:37.891099 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:37.891104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:37.891357 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:37.892064 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:37.892095 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:37.895135 (+ 3040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:37.898965 (+ 3830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:37.898971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:37.898985 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:37.899514 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:37.899927 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:37.899933 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:37.899936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:37.900145 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:37.900711 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:37.900720 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:37.901074 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 19:41:37.901334 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 19:41:37.901893 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":351,"threads_started":1}
I20260807 19:41:37.904687 31364 heartbeater.cc:347] Connected to a master server at 127.14.16.126:40953
I20260807 19:41:37.905020 31364 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:37.905800 31364 heartbeater.cc:511] Master 127.14.16.126:40953 requested a full tablet report, sending...
I20260807 19:41:37.907353 31035 ts_manager.cc:194] Registered new tserver with Master: 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761)
I20260807 19:41:37.908223 31035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:35565
W20260807 19:41:38.241073 31367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:38.241364 31367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:38.241461 31367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:38.249126 31367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:38.249248 31367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:38.249370 31367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:41:38.259261 31367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:42833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=34115
--tserver_master_addrs=127.14.16.126:40953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:41:38.260593 31367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:38.261888 31367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:38.276230 31373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:38.276329 31375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:38.276264 31372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:38.277096 31367 server_base.cc:1034] running on GCE node
I20260807 19:41:38.277778 31367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:38.278885 31367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:38.280141 31367 hybrid_clock.cc:648] HybridClock initialized: now 1786131698280083 us; error 50 us; skew 500 ppm
I20260807 19:41:38.280506 31367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:38 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131698, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:38.283818 31367 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:41:38.285310 31367 webserver.cc:460] Webserver started at http://127.14.16.67:34115/ using document root <none> and password file <none>
I20260807 19:41:38.285974 31367 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:38.286094 31367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:38.291873 31367 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260807 19:41:38.294507 31382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:38.295992 31367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:38.296151 31367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "d62d30da868c4d5d9e4e398f5601d6b3"
format_stamp: "Formatted at 2026-08-07 19:41:15 on dist-test-slave-07c2"
server_key: "a40fbc738e7a836b05b7203771fbdb0c"
server_key_iv: "9f2db8e7cc4d5071c1f89e7fc8ae65fd"
server_key_version: "encryptionkey@0"
I20260807 19:41:38.297171 31367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:38.319339 31367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:38.323098 31367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:38.323370 31367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:38.324360 31367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:41:38.325831 31367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:38.325927 31367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:38.326090 31367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:38.326200 31367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:38.359915 31367 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:42833
I20260807 19:41:38.360003 31499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:42833 every 8 connection(s)
I20260807 19:41:38.361351 31367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:41:38.370110 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 31367
Aug 07 19:41:38 dist-test-slave-07c2 krb5kdc[30436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131698, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:41:38.382668 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.363903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:54643 (local address 127.14.16.126:40953)
0807 19:41:38.364177 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.364182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.365284 (+ 1102us) server_negotiation.cc:407] Connection header received
0807 19:41:38.367018 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.367024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.367137 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.367223 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:38.368798 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.369528 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.371453 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.372038 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.372153 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.376442 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:38.376447 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:38.376451 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:38.376483 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:38.379278 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:38.380082 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:38.380088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:38.380091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:38.380224 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:38.380657 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:38.380663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:38.380666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:38.381025 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:38.381187 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:38.382143 (+ 956us) server_negotiation.cc:299] Negotiation successful
0807 19:41:38.382381 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260807 19:41:38.383924 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.364404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:54643)
0807 19:41:38.365118 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:38.365158 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:38.366770 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:38.367494 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:38.367502 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:38.368053 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:38.368653 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:38.368670 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.369720 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.369725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:38.371231 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:38.371243 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.372318 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.372350 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:38.372667 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.373422 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:38.373461 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:38.376242 (+ 2781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:38.379471 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:38.379479 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:38.379494 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:38.379953 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:38.380337 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:38.380342 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:38.380345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:38.380528 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:38.381178 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:38.381188 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:38.381614 (+ 426us) client_negotiation.cc:769] Sending connection context
0807 19:41:38.381957 (+ 343us) client_negotiation.cc:240] Negotiation successful
0807 19:41:38.382491 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":303,"threads_started":1}
I20260807 19:41:38.385475 31500 heartbeater.cc:347] Connected to a master server at 127.14.16.126:40953
I20260807 19:41:38.385794 31500 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:38.386512 31500 heartbeater.cc:511] Master 127.14.16.126:40953 requested a full tablet report, sending...
I20260807 19:41:38.386675 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.371106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47698 (local address 127.14.16.126:40953)
0807 19:41:38.371655 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.371660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.372025 (+ 365us) server_negotiation.cc:407] Connection header received
0807 19:41:38.372175 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.372180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.372255 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.372371 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:38.374548 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.375689 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.377007 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.377482 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.377634 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.380367 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:38.380375 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:38.380384 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:38.380412 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:38.383815 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:38.384655 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:38.384659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:38.384662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:38.384770 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:38.385225 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:38.385228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:38.385230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:38.385474 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:38.385733 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:38.386080 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 19:41:38.386305 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":290,"threads_started":1}
I20260807 19:41:38.388237 31035 ts_manager.cc:194] Registered new tserver with Master: d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833)
I20260807 19:41:38.389194 31035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:54643
I20260807 19:41:38.389554 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:41:38.395833 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.390527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45800 (local address 127.14.16.126:40953)
0807 19:41:38.390715 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.390723 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.391260 (+ 537us) server_negotiation.cc:407] Connection header received
0807 19:41:38.391498 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.391503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.391663 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.391828 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:38.392164 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.392809 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.394230 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.394623 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.394706 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.395145 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:38.395401 (+ 256us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 19:41:38.395656 (+ 255us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45800: token expired
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 19:41:38.395955 31503 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45800: token expired
W20260807 19:41:38.396502 30954 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.14.16.126:40953)
I20260807 19:41:38.409575 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.396951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45802 (local address 127.14.16.126:40953)
0807 19:41:38.397086 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.397090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.397154 (+ 64us) server_negotiation.cc:407] Connection header received
0807 19:41:38.397348 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.397354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.397422 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.397517 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:38.398928 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.399697 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.401073 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.401461 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.401587 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.404452 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:38.404458 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:38.404462 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:38.404501 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:38.406770 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:38.407569 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:38.407574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:38.407578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:38.407720 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:38.408102 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:38.408108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:38.408112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:38.408420 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:38.408697 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:38.409033 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 19:41:38.409251 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 19:41:38.413199 31035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:41:38.420680 31035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:41:38.449083 31226 heartbeater.cc:503] Master 127.14.16.126:40953 was elected leader, sending a full tablet report...
I20260807 19:41:38.453857 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.432627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48108 (local address 127.14.16.65:36845)
0807 19:41:38.445364 (+ 12737us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.445371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.445396 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:41:38.445481 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.445487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.445707 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.445924 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:38.447018 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.448589 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.451499 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.452668 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.452810 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.453053 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:38.453354 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 19:41:38.453527 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12572,"thread_start_us":12434,"threads_started":1}
I20260807 19:41:38.454782 31512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.433195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:40761 (local address 127.0.0.1:39360)
0807 19:41:38.446282 (+ 13087us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:38.446299 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:38.446458 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:38.447195 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:38.447202 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:38.447230 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:38.447671 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:38.447683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.449561 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.449567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:38.451775 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:38.451790 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.453886 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.453893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:38.454063 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.454093 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:41:38.454172 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 19:41:38.454297 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12971,"mutex_wait_us":1,"thread_start_us":9665,"threads_started":1}
I20260807 19:41:38.455502 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.433317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45696 (local address 127.14.16.67:42833)
0807 19:41:38.444433 (+ 11116us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.444446 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.446144 (+ 1698us) server_negotiation.cc:407] Connection header received
0807 19:41:38.446253 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.446260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.446518 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.446734 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:38.447461 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.449017 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.452581 (+ 3564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.453961 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.454077 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.454946 (+ 869us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:38.455030 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 19:41:38.455191 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10882,"thread_start_us":11640,"threads_started":1}
I20260807 19:41:38.455510 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.433886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39360 (local address 127.14.16.66:40761)
0807 19:41:38.444938 (+ 11052us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.444947 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.446559 (+ 1612us) server_negotiation.cc:407] Connection header received
0807 19:41:38.446637 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.446642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.446878 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.447063 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:38.447852 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.449393 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.451941 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.453401 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.453530 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.453724 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:38.455104 (+ 1380us) server_negotiation.cc:299] Negotiation successful
0807 19:41:38.455223 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10866,"thread_start_us":10718,"threads_started":1}
I20260807 19:41:38.454871 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.432333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:36845 (local address 127.0.0.1:48108)
0807 19:41:38.443396 (+ 11063us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:38.443460 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:38.443684 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:38.446260 (+ 2576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:38.446271 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:38.446299 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:38.446763 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:38.446778 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.448724 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.448732 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:38.451333 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:38.451354 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.452890 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.452897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:38.453096 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.453159 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 19:41:38.453301 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 19:41:38.454466 (+ 1165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10698,"mutex_wait_us":3,"thread_start_us":13005,"threads_started":1}
I20260807 19:41:38.455222 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.432729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:42833 (local address 127.0.0.1:45696)
0807 19:41:38.445594 (+ 12865us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:38.445611 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:38.445725 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:38.446737 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:38.446742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:38.446773 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:38.447223 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:38.447236 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.449184 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.449190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:38.452423 (+ 3233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:38.452438 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.454422 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.454432 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:38.454569 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.454591 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 19:41:38.454648 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 19:41:38.454736 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12738,"mutex_wait_us":49,"thread_start_us":13295,"threads_started":1}
I20260807 19:41:38.459452 31295 tablet_service.cc:1467] Processing CreateTablet for tablet 5c113516eabf47f0af7fd501555376f8 (DEFAULT_TABLE table=test-table [id=20254efde71b429e886185f481b391fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:38.459465 31158 tablet_service.cc:1467] Processing CreateTablet for tablet 5c113516eabf47f0af7fd501555376f8 (DEFAULT_TABLE table=test-table [id=20254efde71b429e886185f481b391fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:38.459865 31434 tablet_service.cc:1467] Processing CreateTablet for tablet 5c113516eabf47f0af7fd501555376f8 (DEFAULT_TABLE table=test-table [id=20254efde71b429e886185f481b391fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:38.460757 31295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c113516eabf47f0af7fd501555376f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:38.460918 31158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c113516eabf47f0af7fd501555376f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:38.461119 31434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c113516eabf47f0af7fd501555376f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:38.474555 31516 tablet_bootstrap.cc:492] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62: Bootstrap starting.
I20260807 19:41:38.479422 31516 tablet_bootstrap.cc:654] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:38.480183 31517 tablet_bootstrap.cc:492] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Bootstrap starting.
I20260807 19:41:38.480921 31516 log.cc:824] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:38.481671 31519 tablet_bootstrap.cc:492] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3: Bootstrap starting.
I20260807 19:41:38.483937 31517 tablet_bootstrap.cc:654] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:38.485164 31519 tablet_bootstrap.cc:654] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:38.485224 31516 tablet_bootstrap.cc:492] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62: No bootstrap required, opened a new log
I20260807 19:41:38.485210 31517 log.cc:824] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:38.485561 31516 ts_tablet_manager.cc:1397] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260807 19:41:38.486459 31519 log.cc:824] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:38.488276 31517 tablet_bootstrap.cc:492] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: No bootstrap required, opened a new log
I20260807 19:41:38.488565 31517 ts_tablet_manager.cc:1397] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:41:38.489789 31519 tablet_bootstrap.cc:492] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3: No bootstrap required, opened a new log
I20260807 19:41:38.490053 31519 ts_tablet_manager.cc:1397] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:41:38.495888 31517 raft_consensus.cc:348] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:38.496510 31517 raft_consensus.cc:374] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:38.496335 31516 raft_consensus.cc:348] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:38.496654 31517 raft_consensus.cc:729] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58dc65b6dca44cd87b7fcb9a1b5bbbe, State: Initialized, Role: FOLLOWER
I20260807 19:41:38.496778 31516 raft_consensus.cc:374] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:38.496891 31516 raft_consensus.cc:729] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be50dc8a3934a58a4ab92f1bef80f62, State: Initialized, Role: FOLLOWER
I20260807 19:41:38.497370 31517 consensus_queue.cc:260] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:38.497558 31516 consensus_queue.cc:260] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:38.498925 31364 heartbeater.cc:503] Master 127.14.16.126:40953 was elected leader, sending a full tablet report...
I20260807 19:41:38.499567 31516 ts_tablet_manager.cc:1428] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260807 19:41:38.501670 31517 ts_tablet_manager.cc:1428] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260807 19:41:38.501848 31519 raft_consensus.cc:348] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:38.502445 31519 raft_consensus.cc:374] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:38.502571 31519 raft_consensus.cc:729] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62d30da868c4d5d9e4e398f5601d6b3, State: Initialized, Role: FOLLOWER
I20260807 19:41:38.503212 31519 consensus_queue.cc:260] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:38.505560 31500 heartbeater.cc:503] Master 127.14.16.126:40953 was elected leader, sending a full tablet report...
I20260807 19:41:38.505748 31519 ts_tablet_manager.cc:1428] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260807 19:41:38.606927 31523 raft_consensus.cc:482] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:41:38.607264 31523 raft_consensus.cc:504] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:38.609055 31523 leader_election.cc:288] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833), 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761)
W20260807 19:41:38.612926 31501 tablet.cc:2369] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:41:38.616469 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.609567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:33203 (local address 127.14.16.66:40761)
0807 19:41:38.609899 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.609903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.610584 (+ 681us) server_negotiation.cc:407] Connection header received
0807 19:41:38.610865 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.610873 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.610946 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.611013 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:38.611799 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.612545 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.614641 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.615530 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.615696 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.615787 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:38.616079 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 19:41:38.616204 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190}
I20260807 19:41:38.616616 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.609341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:40761 (local address 127.14.16.65:33203)
0807 19:41:38.610466 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:38.610486 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:38.610665 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:38.611063 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:38.611067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:38.611092 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:38.611642 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:38.611651 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.612683 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.612691 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:38.614487 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:38.614499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.615740 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.615746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:38.615915 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.615940 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:41:38.616042 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 19:41:38.616177 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"mutex_wait_us":24,"thread_start_us":767,"threads_started":1}
I20260807 19:41:38.616901 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.609553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:60067 (local address 127.14.16.67:42833)
0807 19:41:38.609778 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.609784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.610718 (+ 934us) server_negotiation.cc:407] Connection header received
0807 19:41:38.610902 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.610906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.610966 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.611074 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:38.611890 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.612625 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.615085 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.615970 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.616156 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.616243 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:38.616522 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 19:41:38.616654 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:41:38.617031 31527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.609517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:42833 (local address 127.14.16.65:60067)
0807 19:41:38.610607 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:38.610622 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:38.610738 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:38.611114 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:38.611118 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:38.611146 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:38.611674 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:38.611683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.613133 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.613141 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:38.614921 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:38.614933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.616192 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.616198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:38.616351 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.616373 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 19:41:38.616493 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 19:41:38.616624 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"mutex_wait_us":9,"thread_start_us":745,"threads_started":1}
I20260807 19:41:38.617808 31316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c113516eabf47f0af7fd501555376f8" candidate_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" is_pre_election: true
I20260807 19:41:38.618139 31454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c113516eabf47f0af7fd501555376f8" candidate_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" is_pre_election: true
I20260807 19:41:38.618433 31316 raft_consensus.cc:2454] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a58dc65b6dca44cd87b7fcb9a1b5bbbe in term 0.
I20260807 19:41:38.618645 31454 raft_consensus.cc:2454] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a58dc65b6dca44cd87b7fcb9a1b5bbbe in term 0.
I20260807 19:41:38.619172 31111 leader_election.cc:302] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0be50dc8a3934a58a4ab92f1bef80f62, a58dc65b6dca44cd87b7fcb9a1b5bbbe; no voters:
I20260807 19:41:38.619629 31523 raft_consensus.cc:2781] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:41:38.619781 31523 raft_consensus.cc:482] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:41:38.619892 31523 raft_consensus.cc:3037] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:38.622088 31523 raft_consensus.cc:504] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:41:38.622756 31523 leader_election.cc:288] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 1 election: Requested vote from peers d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833), 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761)
I20260807 19:41:38.623139 31454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c113516eabf47f0af7fd501555376f8" candidate_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62d30da868c4d5d9e4e398f5601d6b3"
I20260807 19:41:38.623304 31316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c113516eabf47f0af7fd501555376f8" candidate_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be50dc8a3934a58a4ab92f1bef80f62"
I20260807 19:41:38.623387 31454 raft_consensus.cc:3037] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:38.623517 31316 raft_consensus.cc:3037] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:38.626663 31454 raft_consensus.cc:2454] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a58dc65b6dca44cd87b7fcb9a1b5bbbe in term 1.
I20260807 19:41:38.626725 31316 raft_consensus.cc:2454] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a58dc65b6dca44cd87b7fcb9a1b5bbbe in term 1.
I20260807 19:41:38.627111 31112 leader_election.cc:302] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a58dc65b6dca44cd87b7fcb9a1b5bbbe, d62d30da868c4d5d9e4e398f5601d6b3; no voters:
I20260807 19:41:38.627439 31523 raft_consensus.cc:2781] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:38.627883 31523 raft_consensus.cc:686] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 LEADER]: Becoming Leader. State: Replica: a58dc65b6dca44cd87b7fcb9a1b5bbbe, State: Running, Role: LEADER
I20260807 19:41:38.628360 31523 consensus_queue.cc:237] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
W20260807 19:41:38.631726 31365 tablet.cc:2369] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:41:38.632968 31035 catalog_manager.cc:5495] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe reported cstate change: term changed from 0 to 1, leader changed from <none> to a58dc65b6dca44cd87b7fcb9a1b5bbbe (127.14.16.65). New cstate: current_term: 1 leader_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } health_report { overall_health: UNKNOWN } } }
I20260807 19:41:38.648748 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.643219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48114 (local address 127.14.16.65:36845)
0807 19:41:38.643456 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.643464 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.643487 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:41:38.643744 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.643748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.643833 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.643990 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:38.644407 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.645139 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.646514 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.646922 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.647054 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.647658 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:38.648142 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:41:38.648310 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:38.648391 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 19:41:38.648469 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 19:41:38.656646 31454 raft_consensus.cc:1261] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3 [term 1 FOLLOWER]: Refusing update from remote peer a58dc65b6dca44cd87b7fcb9a1b5bbbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:41:38.656647 31316 raft_consensus.cc:1261] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 1 FOLLOWER]: Refusing update from remote peer a58dc65b6dca44cd87b7fcb9a1b5bbbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:41:38.657634 31528 consensus_queue.cc:1035] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:38.657974 31523 consensus_queue.cc:1035] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:41:38.663105 31227 tablet.cc:2369] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:41:38.687657 31035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:41:38.687947 31035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:41:38.691713 31035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Sending DeleteTablet for 3 replicas of tablet 5c113516eabf47f0af7fd501555376f8
I20260807 19:41:38.692813 31158 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c113516eabf47f0af7fd501555376f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:38 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:48108
I20260807 19:41:38.693091 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 31073
I20260807 19:41:38.693300 31434 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c113516eabf47f0af7fd501555376f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:38 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:45696
I20260807 19:41:38.694024 31295 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c113516eabf47f0af7fd501555376f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:38 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:39360
I20260807 19:41:38.694144 31543 tablet_replica.cc:330] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3: stopping tablet replica
I20260807 19:41:38.694561 31543 raft_consensus.cc:2229] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:41:38.695127 31543 raft_consensus.cc:2258] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:38.695926 31544 tablet_replica.cc:330] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62: stopping tablet replica
I20260807 19:41:38.696322 31544 raft_consensus.cc:2229] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:41:38.696762 31544 raft_consensus.cc:2258] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:38.697532 31543 ts_tablet_manager.cc:1905] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:38.698639 31544 ts_tablet_manager.cc:1905] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:38.704625 31543 ts_tablet_manager.cc:1918] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:41:38.704782 31543 log.cc:1196] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/5c113516eabf47f0af7fd501555376f8
I20260807 19:41:38.705317 31543 ts_tablet_manager.cc:1939] T 5c113516eabf47f0af7fd501555376f8 P d62d30da868c4d5d9e4e398f5601d6b3: Deleting consensus metadata
I20260807 19:41:38.705550 31544 ts_tablet_manager.cc:1918] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:41:38.705705 31544 log.cc:1196] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/5c113516eabf47f0af7fd501555376f8
I20260807 19:41:38.706265 31544 ts_tablet_manager.cc:1939] T 5c113516eabf47f0af7fd501555376f8 P 0be50dc8a3934a58a4ab92f1bef80f62: Deleting consensus metadata
I20260807 19:41:38.707388 31542 tablet_replica.cc:330] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: stopping tablet replica
I20260807 19:41:38.707376 31020 catalog_manager.cc:4841] TS d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833): tablet 5c113516eabf47f0af7fd501555376f8 (table test-table [id=20254efde71b429e886185f481b391fd]) successfully deleted
I20260807 19:41:38.707842 31542 raft_consensus.cc:2229] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:41:38.707962 31019 catalog_manager.cc:4841] TS 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761): tablet 5c113516eabf47f0af7fd501555376f8 (table test-table [id=20254efde71b429e886185f481b391fd]) successfully deleted
I20260807 19:41:38.708633 31542 raft_consensus.cc:2258] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:38.710739 31542 ts_tablet_manager.cc:1905] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:38.712101 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.693904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48118 (local address 127.14.16.65:36845)
0807 19:41:38.694589 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.694598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.694620 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:41:38.694681 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.694688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.694790 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.694905 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:38.697450 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.698518 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.700526 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.701273 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.701395 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.705058 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:38.705080 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:38.705095 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:38.705127 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:38.708884 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:38.709698 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:38.709706 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:38.709710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:38.709918 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:38.710307 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:38.710312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:38.710315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:38.710707 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:38.711107 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:38.711436 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 19:41:38.711686 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522}
I20260807 19:41:38.712874 31221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:38.811618 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 31073
I20260807 19:41:38.832612 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 31230
W20260807 19:41:38.832700 31018 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.16.65:36845 (error 108)
I20260807 19:41:38.835235 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.834603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:36845 (local address 127.0.0.1:48128)
0807 19:41:38.834864 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:38.835021 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:36845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260807 19:41:38.835525 31018 catalog_manager.cc:4584] TS a58dc65b6dca44cd87b7fcb9a1b5bbbe (127.14.16.65:36845): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c113516eabf47f0af7fd501555376f8: Network error: Client connection negotiation failed: client connection to 127.14.16.65:36845: connect: Connection refused (error 111)
I20260807 19:41:38.851868 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.833645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39374 (local address 127.14.16.66:40761)
0807 19:41:38.833862 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.833868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.833891 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:41:38.834048 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.834054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.834143 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.834229 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:38.836520 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.837285 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.838630 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.839018 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.839116 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.846338 (+ 7222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:38.846356 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:38.846372 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:38.846419 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:38.849287 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:38.849945 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:38.849952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:38.849955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:38.850076 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:38.850408 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:38.850411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:38.850413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:38.850795 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:38.851008 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:38.851368 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 19:41:38.851531 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:41:38.852593 31359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:38.889187 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.888733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:36845 (local address 127.0.0.1:48130)
0807 19:41:38.888962 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:38.889046 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:36845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 19:41:38.890405 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.890041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:36845 (local address 127.0.0.1:48138)
0807 19:41:38.890195 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:38.890296 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:36845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 19:41:38.942060 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 31230
I20260807 19:41:38.950702 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.950235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:36845 (local address 127.0.0.1:48146)
0807 19:41:38.950461 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:38.950543 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:36845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 19:41:38.951792 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.951460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:36845 (local address 127.0.0.1:48156)
0807 19:41:38.951610 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:38.951676 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:36845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 19:41:38.961983 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 31367
I20260807 19:41:38.975221 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:38.962736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45702 (local address 127.14.16.67:42833)
0807 19:41:38.962908 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:38.962912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:38.962961 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:41:38.963118 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:38.963122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:38.963199 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:38.963342 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:38.964646 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.965526 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.966794 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:38.967193 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:38.967360 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:38.970085 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:38.970101 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:38.970117 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:38.970146 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:38.972850 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:38.973537 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:38.973544 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:38.973546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:38.973679 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:38.973988 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:38.973991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:38.973993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:38.974330 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:38.974546 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:38.974802 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 19:41:38.974981 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:41:38.976042 31495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:39.032524 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:39.032054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:36845 (local address 127.0.0.1:48168)
0807 19:41:39.032269 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:39.032368 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:36845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 19:41:39.033627 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:39.033228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:36845 (local address 127.0.0.1:48184)
0807 19:41:39.033445 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:39.033517 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:36845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 19:41:39.067444 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 31367
I20260807 19:41:39.086823 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 31001
I20260807 19:41:39.088085 31066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:39.204110 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 31001
I20260807 19:41:59.229084 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:40953
--webserver_interface=127.14.16.126
--webserver_port=33523
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:40953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:59.572737 31550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:59.573220 31550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:59.573388 31550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:59.582849 31550 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 19:41:59.582954 31550 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 19:41:59.583019 31550 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 19:41:59.583096 31550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:41:59.583195 31550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:59.583317 31550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:59.583407 31550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:41:59.583488 31550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:41:59.593834 31550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:40953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:40953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=33523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:41:59.595209 31550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:59.596599 31550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:59.609021 31558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:59.609090 31556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:59.609153 31555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:59.609922 31550 server_base.cc:1034] running on GCE node
I20260807 19:41:59.610705 31550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:59.612160 31550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:59.613399 31550 hybrid_clock.cc:648] HybridClock initialized: now 1786131719613321 us; error 54 us; skew 500 ppm
I20260807 19:41:59.613740 31550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:59 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131719, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:59.621616 31550 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:41:59.623229 31550 webserver.cc:460] Webserver started at http://127.14.16.126:33523/ using document root <none> and password file <none>
I20260807 19:41:59.623960 31550 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:59.624100 31550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:59.629535 31550 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 19:41:59.632339 31566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:59.633658 31550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:41:59.633817 31550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "a0836839bd76466ea0d415b805f54402"
format_stamp: "Formatted at 2026-08-07 19:41:13 on dist-test-slave-07c2"
server_key: "c8b191c2feb69587714099efee7fcaa4"
server_key_iv: "60eade3f8790fead05757bc88d53c2f6"
server_key_version: "encryptionkey@0"
I20260807 19:41:59.634730 31550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:59.657682 31550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:59.661966 31550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:59.662204 31550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:59.680832 31550 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:40953
I20260807 19:41:59.680863 31617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:40953 every 8 connection(s)
I20260807 19:41:59.682343 31550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:41:59.689277 31618 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:41:59.690208 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 31550
I20260807 19:41:59.691066 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:36845
--local_ip_for_outbound_sockets=127.14.16.65
--tserver_master_addrs=127.14.16.126:40953
--webserver_port=40677
--webserver_interface=127.14.16.65
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:41:59.692785 31618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Bootstrap starting.
I20260807 19:41:59.717118 31618 log.cc:824] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:59.739941 31618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:41:59.740640 31618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Bootstrap complete.
I20260807 19:41:59.754226 31618 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } }
I20260807 19:41:59.755623 31618 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0836839bd76466ea0d415b805f54402, State: Initialized, Role: FOLLOWER
I20260807 19:41:59.756418 31618 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } }
I20260807 19:41:59.756705 31618 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:41:59.756845 31618 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:41:59.757035 31618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:41:59.763144 31618 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } }
I20260807 19:41:59.763778 31618 leader_election.cc:302] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0836839bd76466ea0d415b805f54402; no voters:
I20260807 19:41:59.764525 31618 leader_election.cc:288] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 19:41:59.764685 31623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:41:59.766073 31623 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [term 3 LEADER]: Becoming Leader. State: Replica: a0836839bd76466ea0d415b805f54402, State: Running, Role: LEADER
I20260807 19:41:59.766637 31623 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } }
I20260807 19:41:59.766903 31618 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:41:59.769354 31625 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0836839bd76466ea0d415b805f54402. Latest consensus state: current_term: 3 leader_uuid: "a0836839bd76466ea0d415b805f54402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } } }
I20260807 19:41:59.769765 31625 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:59.769348 31624 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a0836839bd76466ea0d415b805f54402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0836839bd76466ea0d415b805f54402" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40953 } } }
I20260807 19:41:59.770131 31624 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402 [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:59.777524 31629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:41:59.786526 31629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:41:59.788194 31629 catalog_manager.cc:1259] Loaded cluster ID: ecbff516ce8e4e208780f4b1af772a6b
I20260807 19:41:59.788334 31629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:41:59.792549 31629 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:41:59.794226 31629 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Loaded TSK: 0
I20260807 19:41:59.794409 31629 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a0836839bd76466ea0d415b805f54402: Loaded TSK: 1
F20260807 19:41:59.795544 31629 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3c8e2c99ad google::LogMessage::Fail() at ??:0
@ 0x7f3c8e2cc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f3c8e2c94de google::LogMessage::Flush() at ??:0
@ 0x7f3c8e2caa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3c9202d7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3c9201c73e kudu::security::TokenSigner::ImportKeys() at ??:0
I20260807 19:41:59.812839 31641 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f3ca3cf81c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3ca3cf7783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3ca3d832a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f3ca3d830c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f3ca3dcd5e9 std::function<>::operator()() at ??:0
@ 0x7f3ca3cfbcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f3ca3cf9c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3ca3d81610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f3ca3d81439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3c90f15345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3c90f1891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3c90f18799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3c90ef3eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3c941cd6db start_thread at ??:0
@ 0x7f3c8d16361f clone at ??:0
W20260807 19:42:00.134671 31621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:00.134974 31621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:00.135076 31621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:00.143249 31621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:00.143399 31621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:00.143529 31621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:42:00.153627 31621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:36845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=40677
--tserver_master_addrs=127.14.16.126:40953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:00.154870 31621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:00.156219 31621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:00.169454 31671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:00.169617 31669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:00.169617 31668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:00.170608 31621 server_base.cc:1034] running on GCE node
I20260807 19:42:00.171211 31621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:00.172217 31621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:00.173477 31621 hybrid_clock.cc:648] HybridClock initialized: now 1786131720173427 us; error 56 us; skew 500 ppm
I20260807 19:42:00.173825 31621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:00 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131720, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:00.176994 31621 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:42:00.178462 31621 webserver.cc:460] Webserver started at http://127.14.16.65:40677/ using document root <none> and password file <none>
I20260807 19:42:00.179118 31621 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:00.179270 31621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:00.184517 31621 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:42:00.187441 31678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:00.188719 31621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:00.188885 31621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe"
format_stamp: "Formatted at 2026-08-07 19:41:14 on dist-test-slave-07c2"
server_key: "04290f5f6bf8c4e37982f98d3f4abca9"
server_key_iv: "afd7c31d13746a67d2a7a320a6ece99a"
server_key_version: "encryptionkey@0"
I20260807 19:42:00.189833 31621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:00.218091 31621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:00.222558 31621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:00.222851 31621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:00.223884 31621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:00.225711 31686 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 19:42:00.231366 31621 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 19:42:00.231500 31621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260807 19:42:00.231642 31621 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 19:42:00.234627 31621 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 19:42:00.234735 31621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260807 19:42:00.235047 31686 tablet_bootstrap.cc:492] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Bootstrap starting.
I20260807 19:42:00.243256 31686 log.cc:824] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:00.259229 31686 tablet_bootstrap.cc:492] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:42:00.260227 31686 tablet_bootstrap.cc:492] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Bootstrap complete.
I20260807 19:42:00.260839 31686 ts_tablet_manager.cc:1397] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Time spent bootstrapping tablet: real 0.026s user 0.016s sys 0.008s
I20260807 19:42:00.271251 31686 raft_consensus.cc:348] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:42:00.272886 31686 raft_consensus.cc:729] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58dc65b6dca44cd87b7fcb9a1b5bbbe, State: Initialized, Role: FOLLOWER
I20260807 19:42:00.273897 31686 consensus_queue.cc:260] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:42:00.275990 31686 ts_tablet_manager.cc:1428] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe: Time spent starting tablet: real 0.015s user 0.006s sys 0.008s
I20260807 19:42:00.278045 31621 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:36845
I20260807 19:42:00.278095 31800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:36845 every 8 connection(s)
I20260807 19:42:00.279224 31621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:00.280649 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 31621
I20260807 19:42:00.281482 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:40761
--local_ip_for_outbound_sockets=127.14.16.66
--tserver_master_addrs=127.14.16.126:40953
--webserver_port=36489
--webserver_interface=127.14.16.66
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:42:00.284799 31803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:00.282358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:43055)
0807 19:42:00.283340 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:00.283677 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":351,"threads_started":1}
W20260807 19:42:00.285503 31801 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:40953 (0 consecutive failures): Network error: Failed to ping master at 127.14.16.126:40953: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
I20260807 19:42:00.286458 31803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:00.286033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:48913)
0807 19:42:00.286216 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:00.286314 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 19:42:00.287400 31803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:00.286963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:59973)
0807 19:42:00.287128 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:00.287223 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 19:42:00.287652 31801 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:42:00.618794 31804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:00.619081 31804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:00.619248 31804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:00.626591 31804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:00.626772 31804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:00.626971 31804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:42:00.636981 31804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:40761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=36489
--tserver_master_addrs=127.14.16.126:40953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:00.638317 31804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:00.639772 31804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:00.652999 31809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:00.653031 31810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:00.653134 31812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:00.653817 31804 server_base.cc:1034] running on GCE node
I20260807 19:42:00.654361 31804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:00.655225 31804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:00.656471 31804 hybrid_clock.cc:648] HybridClock initialized: now 1786131720656414 us; error 53 us; skew 500 ppm
I20260807 19:42:00.656759 31804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:00 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131720, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:00.659874 31804 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:42:00.661288 31804 webserver.cc:460] Webserver started at http://127.14.16.66:36489/ using document root <none> and password file <none>
I20260807 19:42:00.661970 31804 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:00.662107 31804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:00.667522 31804 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:42:00.670231 31819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:00.671625 31804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:00.671810 31804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "0be50dc8a3934a58a4ab92f1bef80f62"
format_stamp: "Formatted at 2026-08-07 19:41:14 on dist-test-slave-07c2"
server_key: "0c6727f6372c2b9b1bd7933643c5d45e"
server_key_iv: "78eceff018d1fa7bebc17c2de9839fdd"
server_key_version: "encryptionkey@0"
I20260807 19:42:00.672847 31804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:00.697149 31804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:00.701287 31804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:00.701575 31804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:00.702616 31804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:00.704034 31804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:00.704164 31804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:00.704299 31804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:00.704396 31804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:00.738451 31804 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:40761
I20260807 19:42:00.738543 31934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:40761 every 8 connection(s)
I20260807 19:42:00.740011 31804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:00.741741 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 31804
I20260807 19:42:00.742341 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:42833
--local_ip_for_outbound_sockets=127.14.16.67
--tserver_master_addrs=127.14.16.126:40953
--webserver_port=34115
--webserver_interface=127.14.16.67
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:42:00.745272 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:00.742946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:42355)
0807 19:42:00.743834 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:00.744166 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":329,"threads_started":1}
W20260807 19:42:00.746001 31935 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:40953 (0 consecutive failures): Network error: Failed to ping master at 127.14.16.126:40953: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
I20260807 19:42:00.746934 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:00.746504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:56305)
0807 19:42:00.746688 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:00.746784 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 19:42:00.747908 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:00.747496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:59423)
0807 19:42:00.747663 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:00.747761 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 19:42:00.748211 31935 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:42:01.077982 31938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:01.078325 31938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:01.078502 31938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:01.086827 31938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:01.086966 31938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:01.087085 31938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:42:01.097491 31938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:42833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=34115
--tserver_master_addrs=127.14.16.126:40953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:01.098976 31938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:01.100405 31938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:01.113373 31944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:01.113410 31946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:01.114100 31943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:01.114475 31938 server_base.cc:1034] running on GCE node
I20260807 19:42:01.115075 31938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:01.116014 31938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:01.117233 31938 hybrid_clock.cc:648] HybridClock initialized: now 1786131721117177 us; error 54 us; skew 500 ppm
I20260807 19:42:01.117542 31938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:01 dist-test-slave-07c2 krb5kdc[30436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:01.120770 31938 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:42:01.122267 31938 webserver.cc:460] Webserver started at http://127.14.16.67:34115/ using document root <none> and password file <none>
I20260807 19:42:01.122934 31938 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:01.123083 31938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:01.128312 31938 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:01.130868 31953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:01.132388 31938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:42:01.132575 31938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "d62d30da868c4d5d9e4e398f5601d6b3"
format_stamp: "Formatted at 2026-08-07 19:41:15 on dist-test-slave-07c2"
server_key: "a40fbc738e7a836b05b7203771fbdb0c"
server_key_iv: "9f2db8e7cc4d5071c1f89e7fc8ae65fd"
server_key_version: "encryptionkey@0"
I20260807 19:42:01.133597 31938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:01.149744 31938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:01.153860 31938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:01.154151 31938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:01.155169 31938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:01.156503 31938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:01.156641 31938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:01.156774 31938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:01.156875 31938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:01.191417 31938 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:42833
I20260807 19:42:01.192533 32066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:42833 every 8 connection(s)
I20260807 19:42:01.193704 31938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:01.200254 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:01.197718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:59977)
0807 19:42:01.198664 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:01.199035 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":377,"threads_started":1}
W20260807 19:42:01.201145 32067 heartbeater.cc:644] Failed to heartbeat to 127.14.16.126:40953 (0 consecutive failures): Network error: Failed to ping master at 127.14.16.126:40953: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
I20260807 19:42:01.202243 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:01.201778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:59071)
0807 19:42:01.201945 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:01.202055 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 19:42:01.203560 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:01.202929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:53567)
0807 19:42:01.203279 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:01.203408 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236}
I20260807 19:42:01.203768 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 31938
W20260807 19:42:01.203835 32067 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:42:01.288966 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:01.288151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:37405)
0807 19:42:01.288650 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:01.288744 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":235,"threads_started":1}
I20260807 19:42:01.492477 32072 raft_consensus.cc:482] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:01.492914 32072 raft_consensus.cc:504] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:42:01.495179 32072 leader_election.cc:288] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833), 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761)
Aug 07 19:42:01 dist-test-slave-07c2 krb5kdc[30436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131720, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
Aug 07 19:42:01 dist-test-slave-07c2 krb5kdc[30436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131720, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:42:01.517552 32074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:01.496026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:35229 (local address 127.14.16.67:42833)
0807 19:42:01.497437 (+ 1411us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:01.497448 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:01.497491 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:42:01.499251 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:01.499260 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:01.499668 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:01.499980 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:01.501656 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:01.502858 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:01.504642 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:01.505275 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:01.505483 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:01.510203 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:01.510210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:01.510224 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:01.510270 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:01.513437 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:01.514505 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:01.514512 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:01.514517 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:01.514673 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:01.515111 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:01.515116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:01.515121 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:01.515712 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:01.515924 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:01.516759 (+ 835us) server_negotiation.cc:299] Negotiation successful
0807 19:42:01.517150 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1237,"thread_start_us":1065,"threads_started":1}
I20260807 19:42:01.517536 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:01.495466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:42833 (local address 127.14.16.65:35229)
0807 19:42:01.495857 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:01.495904 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:01.496505 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:01.500014 (+ 3509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:01.500025 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:01.500805 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:01.501428 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:01.501450 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:01.503068 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:01.503072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:01.504494 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:01.504507 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:01.505518 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:01.505526 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:01.505818 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:01.506701 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:01.506731 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:01.509939 (+ 3208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:01.513585 (+ 3646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:01.513594 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:01.513615 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:01.514353 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:01.514797 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:01.514803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:01.514806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:01.514992 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:01.515860 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:01.515868 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:01.516229 (+ 361us) client_negotiation.cc:769] Sending connection context
0807 19:42:01.516497 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 19:42:01.517064 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":179,"spinlock_wait_cycles":29696}
I20260807 19:42:01.517791 32073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:01.495494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:40761 (local address 127.14.16.65:50555)
0807 19:42:01.496639 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:01.496662 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:01.496810 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:01.500015 (+ 3205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:01.500026 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:01.500891 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:01.501523 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:01.501537 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:01.503095 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:01.503099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:01.504502 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:01.504515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:01.505528 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:01.505536 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:01.505823 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:01.506721 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:01.506738 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:01.509649 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:01.513620 (+ 3971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:01.513658 (+ 38us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:01.513662 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:01.514306 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:01.514766 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:01.514772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:01.514775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:01.514999 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:01.515859 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:01.515867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:01.516227 (+ 360us) client_negotiation.cc:769] Sending connection context
0807 19:42:01.516604 (+ 377us) client_negotiation.cc:240] Negotiation successful
0807 19:42:01.517255 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":368,"spinlock_wait_cycles":3072,"thread_start_us":758,"threads_started":1}
I20260807 19:42:01.517841 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:01.496363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:50555 (local address 127.14.16.66:40761)
0807 19:42:01.497440 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:01.497451 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:01.497486 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:42:01.498955 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:01.498965 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:01.499508 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:01.499864 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:01.501747 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:01.502903 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:01.504652 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:01.505291 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:01.505529 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:01.509975 (+ 4446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:01.509984 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:01.509999 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:01.510046 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:01.513353 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:01.514450 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:01.514461 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:01.514473 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:01.514643 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:01.515116 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:01.515121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:01.515146 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:01.515709 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:01.515973 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:01.516974 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0807 19:42:01.517391 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":701,"threads_started":1}
W20260807 19:42:01.521135 31680 leader_election.cc:341] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761): Not found: Tablet not found: 5c113516eabf47f0af7fd501555376f8
W20260807 19:42:01.521473 31682 leader_election.cc:341] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833): Not found: Tablet not found: 5c113516eabf47f0af7fd501555376f8
I20260807 19:42:01.521636 31682 leader_election.cc:302] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a58dc65b6dca44cd87b7fcb9a1b5bbbe; no voters: 0be50dc8a3934a58a4ab92f1bef80f62, d62d30da868c4d5d9e4e398f5601d6b3
I20260807 19:42:01.522082 32072 raft_consensus.cc:2726] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:42:01.749590 32077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:01.748751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:59047)
0807 19:42:01.749217 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:01.749355 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":212,"threads_started":1}
I20260807 19:42:02.205375 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:02.204460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:57455)
0807 19:42:02.204999 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:02.205167 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":295,"threads_started":1}
I20260807 19:42:02.290804 32079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:02.290033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:39827)
0807 19:42:02.290530 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:02.290636 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":261,"threads_started":1}
I20260807 19:42:02.751174 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:02.750485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:51589)
0807 19:42:02.750937 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:02.751021 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":253,"threads_started":1}
I20260807 19:42:03.207070 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:03.206230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:48295)
0807 19:42:03.206806 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:03.206884 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":330,"threads_started":1}
I20260807 19:42:03.292433 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:03.291728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:60745)
0807 19:42:03.292184 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:03.292265 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":221,"threads_started":1}
I20260807 19:42:03.709209 32083 raft_consensus.cc:482] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:03.709448 32083 raft_consensus.cc:504] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:42:03.710104 32083 leader_election.cc:288] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833), 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761)
W20260807 19:42:03.711242 31682 leader_election.cc:341] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833): Not found: Tablet not found: 5c113516eabf47f0af7fd501555376f8
W20260807 19:42:03.711460 31680 leader_election.cc:341] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761): Not found: Tablet not found: 5c113516eabf47f0af7fd501555376f8
I20260807 19:42:03.711586 31680 leader_election.cc:302] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a58dc65b6dca44cd87b7fcb9a1b5bbbe; no voters: 0be50dc8a3934a58a4ab92f1bef80f62, d62d30da868c4d5d9e4e398f5601d6b3
I20260807 19:42:03.711882 32083 raft_consensus.cc:2726] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:42:03.758424 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:03.755648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:52855)
0807 19:42:03.758048 (+ 2400us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:03.758182 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2215,"thread_start_us":2124,"threads_started":1}
I20260807 19:42:04.208711 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:04.207950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:45493)
0807 19:42:04.208418 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:04.208515 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":238,"threads_started":1}
I20260807 19:42:04.294091 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:04.293401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:34553)
0807 19:42:04.293853 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:04.293932 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":258,"threads_started":1}
I20260807 19:42:04.760136 32088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:04.759403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:56901)
0807 19:42:04.759875 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:04.759953 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":258,"threads_started":1}
I20260807 19:42:05.210340 32089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:05.209612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:45417)
0807 19:42:05.210047 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:05.210134 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":224,"threads_started":1}
I20260807 19:42:05.295961 32090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:05.295050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:34867)
0807 19:42:05.295655 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:05.295757 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":308,"threads_started":1}
I20260807 19:42:05.761780 32091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:05.761044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:37307)
0807 19:42:05.761512 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:05.761592 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":234,"threads_started":1}
I20260807 19:42:05.970711 32092 raft_consensus.cc:482] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:05.970960 32092 raft_consensus.cc:504] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:42:05.971889 32092 leader_election.cc:288] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833), 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761)
W20260807 19:42:05.972738 31682 leader_election.cc:341] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833): Not found: Tablet not found: 5c113516eabf47f0af7fd501555376f8
W20260807 19:42:05.972927 31680 leader_election.cc:341] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761): Not found: Tablet not found: 5c113516eabf47f0af7fd501555376f8
I20260807 19:42:05.973052 31680 leader_election.cc:302] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a58dc65b6dca44cd87b7fcb9a1b5bbbe; no voters: 0be50dc8a3934a58a4ab92f1bef80f62, d62d30da868c4d5d9e4e398f5601d6b3
I20260807 19:42:05.973277 32092 raft_consensus.cc:2726] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:42:06.212173 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:06.211409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:34647)
0807 19:42:06.211906 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:06.211997 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":266,"threads_started":1}
I20260807 19:42:06.297843 32094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:06.296930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:50029)
0807 19:42:06.297558 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:06.297662 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":339,"threads_started":1}
I20260807 19:42:06.763617 32095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:06.762775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:55379)
0807 19:42:06.763265 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:06.763424 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":258,"threads_started":1}
I20260807 19:42:07.214165 32096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:07.213202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:33249)
0807 19:42:07.213827 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:07.213940 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":339,"threads_started":1}
I20260807 19:42:07.299736 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:07.298868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:42725)
0807 19:42:07.299432 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:07.299523 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":264,"threads_started":1}
I20260807 19:42:07.765807 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:07.764758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:51927)
0807 19:42:07.765389 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:07.765530 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":335,"threads_started":1}
I20260807 19:42:07.886282 32100 raft_consensus.cc:482] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:07.886531 32100 raft_consensus.cc:504] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:42:07.887225 32100 leader_election.cc:288] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833), 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761)
W20260807 19:42:07.888170 31682 leader_election.cc:341] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833): Not found: Tablet not found: 5c113516eabf47f0af7fd501555376f8
W20260807 19:42:07.888343 31680 leader_election.cc:341] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761): Not found: Tablet not found: 5c113516eabf47f0af7fd501555376f8
I20260807 19:42:07.888515 31680 leader_election.cc:302] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a58dc65b6dca44cd87b7fcb9a1b5bbbe; no voters: 0be50dc8a3934a58a4ab92f1bef80f62, d62d30da868c4d5d9e4e398f5601d6b3
I20260807 19:42:07.888782 32100 raft_consensus.cc:2726] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:42:08.216233 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:08.215208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:35745)
0807 19:42:08.215845 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:08.215935 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":271,"threads_started":1}
I20260807 19:42:08.301576 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:08.300776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:40751)
0807 19:42:08.301292 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:08.301383 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":271,"threads_started":1}
I20260807 19:42:08.767616 32103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:08.766754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:41861)
0807 19:42:08.767279 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:08.767428 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":310,"threads_started":1}
I20260807 19:42:09.217984 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:09.217292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:45719)
0807 19:42:09.217718 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:09.217810 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":235,"threads_started":1}
I20260807 19:42:09.303550 32105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:09.302591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:32793)
0807 19:42:09.303181 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:09.303346 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":343,"threads_started":1}
I20260807 19:42:09.769368 32106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:09.768602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:58361)
0807 19:42:09.769123 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:09.769210 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":244,"threads_started":1}
I20260807 19:42:10.219740 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:10.218916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:48733)
0807 19:42:10.219473 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:10.219566 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":246,"threads_started":1}
I20260807 19:42:10.305472 32108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:10.304690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:56633)
0807 19:42:10.305225 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:10.305306 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":288,"threads_started":1}
I20260807 19:42:10.771035 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:10.770342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:56159)
0807 19:42:10.770775 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:10.770855 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":224,"threads_started":1}
I20260807 19:42:11.221577 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:11.220719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:56773)
0807 19:42:11.221316 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:11.221392 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":388,"threads_started":1}
I20260807 19:42:11.307060 32111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:11.306363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:50335)
0807 19:42:11.306794 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:11.306889 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":218,"threads_started":1}
I20260807 19:42:11.772792 32112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:11.771991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:43271)
0807 19:42:11.772508 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:11.772613 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":254,"threads_started":1}
I20260807 19:42:12.223264 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:12.222569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:33997)
0807 19:42:12.222998 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:12.223083 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":224,"threads_started":1}
I20260807 19:42:12.308853 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:12.308063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:40461)
0807 19:42:12.308565 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:12.308659 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":273,"threads_started":1}
I20260807 19:42:12.774587 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:12.773734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:58669)
0807 19:42:12.774319 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:12.774406 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":292,"threads_started":1}
I20260807 19:42:13.225293 32116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:13.224402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:59079)
0807 19:42:13.224944 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:13.225031 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":293,"threads_started":1}
I20260807 19:42:13.310678 32117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:13.309862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:37279)
0807 19:42:13.310395 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:13.310496 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":284,"threads_started":1}
I20260807 19:42:13.776439 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:13.775607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:57437)
0807 19:42:13.776171 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:13.776272 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":329,"threads_started":1}
I20260807 19:42:13.815119 32119 raft_consensus.cc:482] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:13.815384 32119 raft_consensus.cc:504] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58dc65b6dca44cd87b7fcb9a1b5bbbe" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 36845 } } peers { permanent_uuid: "d62d30da868c4d5d9e4e398f5601d6b3" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 42833 } } peers { permanent_uuid: "0be50dc8a3934a58a4ab92f1bef80f62" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 40761 } }
I20260807 19:42:13.816308 32119 leader_election.cc:288] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833), 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761)
W20260807 19:42:13.817205 31682 leader_election.cc:341] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d62d30da868c4d5d9e4e398f5601d6b3 (127.14.16.67:42833): Not found: Tablet not found: 5c113516eabf47f0af7fd501555376f8
W20260807 19:42:13.817387 31680 leader_election.cc:341] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0be50dc8a3934a58a4ab92f1bef80f62 (127.14.16.66:40761): Not found: Tablet not found: 5c113516eabf47f0af7fd501555376f8
I20260807 19:42:13.817577 31680 leader_election.cc:302] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a58dc65b6dca44cd87b7fcb9a1b5bbbe; no voters: 0be50dc8a3934a58a4ab92f1bef80f62, d62d30da868c4d5d9e4e398f5601d6b3
I20260807 19:42:13.817845 32119 raft_consensus.cc:2726] T 5c113516eabf47f0af7fd501555376f8 P a58dc65b6dca44cd87b7fcb9a1b5bbbe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:42:14.227164 32120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:14.226354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:49737)
0807 19:42:14.226823 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:14.226915 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":231,"threads_started":1}
I20260807 19:42:14.312611 32121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:14.311743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:34675)
0807 19:42:14.312230 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:14.312371 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":240,"threads_started":1}
I20260807 19:42:14.778105 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:14.777352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:56713)
0807 19:42:14.777806 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:14.777901 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":272,"threads_started":1}
I20260807 19:42:15.228996 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:15.228104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:60751)
0807 19:42:15.228692 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:15.228784 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":349,"threads_started":1}
I20260807 19:42:15.314642 32124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:15.313886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.65:51901)
0807 19:42:15.314333 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:15.314469 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":230,"threads_started":1}
I20260807 19:42:15.779956 32126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:15.779167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.66:39593)
0807 19:42:15.779650 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:15.779787 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":274,"threads_started":1}
I20260807 19:42:16.212723 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 31621
I20260807 19:42:16.228577 32127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:16.214240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.14.16.65:36845)
0807 19:42:16.214910 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:16.214919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:16.214978 (+ 59us) server_negotiation.cc:407] Connection header received
0807 19:42:16.215066 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:16.215078 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:16.215425 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:16.215568 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:16.217126 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.218203 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.219503 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.220115 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.220234 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:16.222894 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:16.222906 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:16.222920 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:16.222958 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:16.225984 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.226634 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.226641 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.226644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.226772 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.227087 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.227090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.227092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.227498 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:16.227742 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:16.227974 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 19:42:16.228170 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":255,"threads_started":1}
I20260807 19:42:16.229911 31795 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:16.231187 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:16.230266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40953 (local address 127.14.16.67:43655)
0807 19:42:16.230820 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:16.230968 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.126:40953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":255,"threads_started":1}
I20260807 19:42:16.322022 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 31621
I20260807 19:42:16.346709 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 31804
I20260807 19:42:16.368896 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:16.347608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48962 (local address 127.14.16.66:40761)
0807 19:42:16.348209 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:16.348214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:16.348233 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:42:16.348471 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:16.348475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:16.348560 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:16.348728 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:16.350872 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.352086 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.354247 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.354713 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.354809 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:16.362988 (+ 8179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:16.362993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:16.362997 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:16.363030 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:16.365875 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.366850 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.366854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.366856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.367002 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.367462 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.367465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.367468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.367747 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:16.367949 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:16.368346 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 19:42:16.368513 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":797,"threads_started":1}
I20260807 19:42:16.369652 31930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:16.455451 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 31804
I20260807 19:42:16.475533 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 31938
I20260807 19:42:16.489866 32133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:16.476306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58812 (local address 127.14.16.67:42833)
0807 19:42:16.476869 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:16.476874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:16.476889 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:42:16.476944 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:16.476948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:16.477006 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:16.477129 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:16.478596 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.479570 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.480917 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.481375 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.481512 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:16.484334 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:16.484340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:16.484343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:16.484379 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:16.487057 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.487815 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.487821 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.487825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.487969 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.488360 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.488369 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.488371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.488674 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:16.488984 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:16.489258 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 19:42:16.489485 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":270,"threads_started":1}
I20260807 19:42:16.490723 32062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:16.576872 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 31938
2026-08-07T19:42:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64705 ms)
[----------] 30 tests from SecurityITest (300928 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:16 dist-test-slave-07c2 krb5kdc[32137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:16 dist-test-slave-07c2 krb5kdc[32137](info): set up 2 sockets
Aug 07 19:42:16 dist-test-slave-07c2 krb5kdc[32137](info): commencing operation
krb5kdc: starting...
W20260807 19:42:17.730592 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:17 dist-test-slave-07c2 krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:42:18.048784 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:39057
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:39057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:18.404402 32150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:18.404724 32150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:18.404834 32150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:18.412811 32150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:18.412916 32150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:18.412972 32150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:18.413033 32150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:18.413079 32150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:18.423100 32150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:39057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:39057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:18.424399 32150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:18.425693 32150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:18.437556 32157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:18.437639 32156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:18.437618 32159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:18.438251 32150 server_base.cc:1034] running on GCE node
I20260807 19:42:18.439090 32150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:18.440451 32150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:18.441691 32150 hybrid_clock.cc:648] HybridClock initialized: now 1786131738441645 us; error 33 us; skew 500 ppm
I20260807 19:42:18.442050 32150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:18 dist-test-slave-07c2 krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131738, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:18.445308 32150 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:42:18.446839 32150 webserver.cc:460] Webserver started at http://127.14.16.126:38667/ using document root <none> and password file <none>
I20260807 19:42:18.447597 32150 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:18.447748 32150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:18.448058 32150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:18.451100 32150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "09eaa5a0da9c4836ba793ff3c10a729e"
format_stamp: "Formatted at 2026-08-07 19:42:18 on dist-test-slave-07c2"
server_key: "08bb06af00da375d15928e2af9dc6f34"
server_key_iv: "250febf90b429d836321913353b91cb8"
server_key_version: "encryptionkey@0"
I20260807 19:42:18.451910 32150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "09eaa5a0da9c4836ba793ff3c10a729e"
format_stamp: "Formatted at 2026-08-07 19:42:18 on dist-test-slave-07c2"
server_key: "08bb06af00da375d15928e2af9dc6f34"
server_key_iv: "250febf90b429d836321913353b91cb8"
server_key_version: "encryptionkey@0"
I20260807 19:42:18.457006 32150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:18.460338 32166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:18.461510 32150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:18.461694 32150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "09eaa5a0da9c4836ba793ff3c10a729e"
format_stamp: "Formatted at 2026-08-07 19:42:18 on dist-test-slave-07c2"
server_key: "08bb06af00da375d15928e2af9dc6f34"
server_key_iv: "250febf90b429d836321913353b91cb8"
server_key_version: "encryptionkey@0"
I20260807 19:42:18.461884 32150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:18.489308 32150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:18.493743 32150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:18.494053 32150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:18.512912 32150 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:39057
I20260807 19:42:18.512931 32218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:39057 every 8 connection(s)
I20260807 19:42:18.514290 32150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:18.517921 32219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:18.519644 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 32150
I20260807 19:42:18.519855 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:18.520598 14401 external_mini_cluster.cc:1428] Setting key 22912c852af01d773fb8a400d3f6451e
I20260807 19:42:18.529937 32219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e: Bootstrap starting.
I20260807 19:42:18.533690 32219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:18.535022 32219 log.cc:824] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:42:18 dist-test-slave-07c2 krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:42:18.538595 32219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e: No bootstrap required, opened a new log
I20260807 19:42:18.546159 32222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:18.522686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35464 (local address 127.14.16.126:39057)
0807 19:42:18.523575 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:18.523587 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:18.523628 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:42:18.525314 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:18.525333 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:18.525701 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:18.526016 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:18.527884 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:18.529195 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:18.531010 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:18.531742 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:18.531877 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:18.536974 (+ 5097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:18.536986 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:18.537003 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:18.537060 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:18.541183 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:18.541945 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:18.541954 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:18.541963 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:18.542205 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:18.542582 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:18.542592 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:18.542597 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:18.543426 (+ 829us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:18.543753 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:18.544157 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 19:42:18.544671 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":432,"threads_started":1}
I20260807 19:42:18.549551 32219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09eaa5a0da9c4836ba793ff3c10a729e" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39057 } }
I20260807 19:42:18.549908 32219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:18.549986 32219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09eaa5a0da9c4836ba793ff3c10a729e, State: Initialized, Role: FOLLOWER
I20260807 19:42:18.550531 32219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09eaa5a0da9c4836ba793ff3c10a729e" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39057 } }
I20260807 19:42:18.550748 32219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:18.550846 32219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:18.550981 32219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:18.553467 32219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09eaa5a0da9c4836ba793ff3c10a729e" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39057 } }
I20260807 19:42:18.554064 32219 leader_election.cc:302] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09eaa5a0da9c4836ba793ff3c10a729e; no voters:
I20260807 19:42:18.554657 32219 leader_election.cc:288] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:18.554804 32224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:18.555569 32224 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [term 1 LEADER]: Becoming Leader. State: Replica: 09eaa5a0da9c4836ba793ff3c10a729e, State: Running, Role: LEADER
I20260807 19:42:18.556231 32224 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09eaa5a0da9c4836ba793ff3c10a729e" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39057 } }
I20260807 19:42:18.557065 32219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:18.559711 32225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09eaa5a0da9c4836ba793ff3c10a729e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09eaa5a0da9c4836ba793ff3c10a729e" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39057 } } }
I20260807 19:42:18.560177 32225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:18.559854 32226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09eaa5a0da9c4836ba793ff3c10a729e. Latest consensus state: current_term: 1 leader_uuid: "09eaa5a0da9c4836ba793ff3c10a729e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09eaa5a0da9c4836ba793ff3c10a729e" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39057 } } }
I20260807 19:42:18.560467 32226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:18.562333 32235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:18.568576 32235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:18.578081 32235 catalog_manager.cc:1347] Generated new cluster ID: 43fc3a0bba744e329e1630a2c398378c
I20260807 19:42:18.578215 32235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:18.590073 32235 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:18.592079 32235 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:18.602839 32235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e: Generated new TSK 0
I20260807 19:42:18.603794 32235 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:42:18.692660 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39057
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:19.042518 32247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:19.042815 32247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:19.042913 32247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:19.050369 32247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:19.050488 32247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:19.050613 32247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:42:19.060825 32247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:19.062317 32247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:19.063668 32247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:19.077144 32252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:19.077307 32255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:19.077145 32253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:19.077842 32247 server_base.cc:1034] running on GCE node
I20260807 19:42:19.078470 32247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:19.079492 32247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:19.080740 32247 hybrid_clock.cc:648] HybridClock initialized: now 1786131739080695 us; error 49 us; skew 500 ppm
I20260807 19:42:19.081090 32247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:19 dist-test-slave-07c2 krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131739, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:19.084297 32247 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:42:19.085731 32247 webserver.cc:460] Webserver started at http://127.14.16.65:34667/ using document root <none> and password file <none>
I20260807 19:42:19.086441 32247 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:19.086599 32247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:19.086946 32247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:19.090078 32247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "61244b5444c446b589dee698cf1601cb"
format_stamp: "Formatted at 2026-08-07 19:42:19 on dist-test-slave-07c2"
server_key: "ec43fa6d15e376d3c35f797bb8e22f0b"
server_key_iv: "915a1c19606dd3b59f665c90756e2b34"
server_key_version: "encryptionkey@0"
I20260807 19:42:19.090862 32247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "61244b5444c446b589dee698cf1601cb"
format_stamp: "Formatted at 2026-08-07 19:42:19 on dist-test-slave-07c2"
server_key: "ec43fa6d15e376d3c35f797bb8e22f0b"
server_key_iv: "915a1c19606dd3b59f665c90756e2b34"
server_key_version: "encryptionkey@0"
I20260807 19:42:19.096184 32247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 19:42:19.099639 32263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:19.100919 32247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:42:19.101114 32247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "61244b5444c446b589dee698cf1601cb"
format_stamp: "Formatted at 2026-08-07 19:42:19 on dist-test-slave-07c2"
server_key: "ec43fa6d15e376d3c35f797bb8e22f0b"
server_key_iv: "915a1c19606dd3b59f665c90756e2b34"
server_key_version: "encryptionkey@0"
I20260807 19:42:19.101326 32247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:19.124779 32247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:19.128914 32247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:19.129235 32247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:19.130254 32247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:19.131666 32247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:19.131771 32247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:19.131929 32247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:19.132005 32247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:19.167955 32247 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:37147
I20260807 19:42:19.168016 32378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:37147 every 8 connection(s)
I20260807 19:42:19.169328 32247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:19.179637 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 32247
I20260807 19:42:19.179879 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:19.180563 14401 external_mini_cluster.cc:1428] Setting key c669d0473fc95cf9e975535192c80521
Aug 07 19:42:19 dist-test-slave-07c2 krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131739, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:42:19.190410 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:19.171875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:38315 (local address 127.14.16.126:39057)
0807 19:42:19.172358 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:19.172362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:19.173129 (+ 767us) server_negotiation.cc:407] Connection header received
0807 19:42:19.175010 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:19.175016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:19.175107 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:19.175229 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:19.176746 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.177497 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.179079 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.179583 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.179722 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:19.184181 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:19.184187 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:19.184190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:19.184222 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:19.186892 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:19.187935 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:19.187939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:19.187941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:19.188064 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:19.188575 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:19.188579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:19.188581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:19.188841 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:19.188966 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:19.189961 (+ 995us) server_negotiation.cc:299] Negotiation successful
0807 19:42:19.190145 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":267,"threads_started":1}
I20260807 19:42:19.192113 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:19.172287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39057 (local address 127.14.16.65:38315)
0807 19:42:19.172970 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:19.173013 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:19.174708 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:19.175487 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:19.175495 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:19.176008 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:19.176571 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:19.176588 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.177679 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.177683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:19.178929 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:19.178941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.179841 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.179849 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:19.180137 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:19.180845 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:19.180868 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:19.183961 (+ 3093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:19.187054 (+ 3093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:19.187067 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:19.187089 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:19.187817 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:19.188175 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:19.188182 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:19.188188 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:19.188450 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:19.188967 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:19.188978 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:19.189503 (+ 525us) client_negotiation.cc:769] Sending connection context
0807 19:42:19.189843 (+ 340us) client_negotiation.cc:240] Negotiation successful
0807 19:42:19.190492 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":267,"threads_started":1}
I20260807 19:42:19.193878 32379 heartbeater.cc:347] Connected to a master server at 127.14.16.126:39057
I20260807 19:42:19.194326 32379 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:19.195081 32379 heartbeater.cc:511] Master 127.14.16.126:39057 requested a full tablet report, sending...
I20260807 19:42:19.197639 32183 ts_manager.cc:194] Registered new tserver with Master: 61244b5444c446b589dee698cf1601cb (127.14.16.65:37147)
I20260807 19:42:19.199967 32183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:38315
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:42:19.253898 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39057
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:19.604206 32385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:19.604532 32385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:19.604724 32385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:19.612408 32385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:19.612568 32385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:19.612753 32385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:42:19.624356 32385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:19.626156 32385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:19.627913 32385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:19.642629 32390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:19.642673 32391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:19.643278 32393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:19.643844 32385 server_base.cc:1034] running on GCE node
I20260807 19:42:19.644526 32385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:19.645552 32385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:19.646786 32385 hybrid_clock.cc:648] HybridClock initialized: now 1786131739646733 us; error 44 us; skew 500 ppm
I20260807 19:42:19.647114 32385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:19 dist-test-slave-07c2 krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131739, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:19.650449 32385 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:42:19.652137 32385 webserver.cc:460] Webserver started at http://127.14.16.66:43799/ using document root <none> and password file <none>
I20260807 19:42:19.652868 32385 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:19.653000 32385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:19.653342 32385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:19.656443 32385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "332dd4d510604ca5a6f7aae4447610ff"
format_stamp: "Formatted at 2026-08-07 19:42:19 on dist-test-slave-07c2"
server_key: "f6b20554e018e53f163daa4baea7a3f0"
server_key_iv: "d1b7c028d4018419b81baf720c60dce4"
server_key_version: "encryptionkey@0"
I20260807 19:42:19.657231 32385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "332dd4d510604ca5a6f7aae4447610ff"
format_stamp: "Formatted at 2026-08-07 19:42:19 on dist-test-slave-07c2"
server_key: "f6b20554e018e53f163daa4baea7a3f0"
server_key_iv: "d1b7c028d4018419b81baf720c60dce4"
server_key_version: "encryptionkey@0"
I20260807 19:42:19.663071 32385 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:42:19.667054 32400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:19.668596 32385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:19.668794 32385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "332dd4d510604ca5a6f7aae4447610ff"
format_stamp: "Formatted at 2026-08-07 19:42:19 on dist-test-slave-07c2"
server_key: "f6b20554e018e53f163daa4baea7a3f0"
server_key_iv: "d1b7c028d4018419b81baf720c60dce4"
server_key_version: "encryptionkey@0"
I20260807 19:42:19.669008 32385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:19.698167 32385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:19.702839 32385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:19.703155 32385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:19.704223 32385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:19.705775 32385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:19.705875 32385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:19.706022 32385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:19.706090 32385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:19.740824 32385 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:35947
I20260807 19:42:19.740931 32514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:35947 every 8 connection(s)
I20260807 19:42:19.742182 32385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:19.746882 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 32385
I20260807 19:42:19.747081 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:19.747671 14401 external_mini_cluster.cc:1428] Setting key dc982f7eca32cf153c178061848d89da
Aug 07 19:42:19 dist-test-slave-07c2 krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131739, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:42:19.765899 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:19.744921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:47747 (local address 127.14.16.126:39057)
0807 19:42:19.745353 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:19.745358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:19.746332 (+ 974us) server_negotiation.cc:407] Connection header received
0807 19:42:19.748426 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:19.748431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:19.748523 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:19.748675 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:19.750429 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.751262 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.753816 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.754297 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.754408 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:19.759011 (+ 4603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:19.759019 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:19.759023 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:19.759073 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:19.762128 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:19.762999 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:19.763006 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:19.763012 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:19.763175 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:19.763698 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:19.763706 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:19.763709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:19.764021 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:19.764200 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:19.765090 (+ 890us) server_negotiation.cc:299] Negotiation successful
0807 19:42:19.765366 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":216,"threads_started":1}
I20260807 19:42:19.766948 32519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:19.745470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39057 (local address 127.14.16.66:47747)
0807 19:42:19.746173 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:19.746216 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:19.748148 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:19.748920 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:19.748929 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:19.749474 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:19.750229 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:19.750257 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.751454 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.751462 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:19.753646 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:19.753665 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.754566 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.754576 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:19.754909 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:19.755738 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:19.755773 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:19.758737 (+ 2964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:19.762306 (+ 3569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:19.762317 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:19.762338 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:19.762855 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:19.763330 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:19.763335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:19.763340 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:19.763561 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:19.764180 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:19.764190 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:19.764633 (+ 443us) client_negotiation.cc:769] Sending connection context
0807 19:42:19.764971 (+ 338us) client_negotiation.cc:240] Negotiation successful
0807 19:42:19.765617 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":286,"threads_started":1}
I20260807 19:42:19.768599 32516 heartbeater.cc:347] Connected to a master server at 127.14.16.126:39057
I20260807 19:42:19.768960 32516 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:19.769874 32516 heartbeater.cc:511] Master 127.14.16.126:39057 requested a full tablet report, sending...
I20260807 19:42:19.771749 32183 ts_manager.cc:194] Registered new tserver with Master: 332dd4d510604ca5a6f7aae4447610ff (127.14.16.66:35947)
I20260807 19:42:19.772709 32183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:47747
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:42:19.822171 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39057
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:20.166322 32522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:20.166644 32522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:20.166812 32522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:20.176587 32522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:20.176697 32522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:20.176817 32522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:42:20.186473 32522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:39057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:20.187767 32522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:20.189015 32522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:20.202728 32530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:20.202692 32527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:20.202796 32528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:20.203529 32522 server_base.cc:1034] running on GCE node
I20260807 19:42:20.204107 32379 heartbeater.cc:503] Master 127.14.16.126:39057 was elected leader, sending a full tablet report...
I20260807 19:42:20.204269 32522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:20.205489 32522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:20.206745 32522 hybrid_clock.cc:648] HybridClock initialized: now 1786131740206654 us; error 83 us; skew 500 ppm
I20260807 19:42:20.207070 32522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:20 dist-test-slave-07c2 krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131740, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:20.210299 32522 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:42:20.211897 32522 webserver.cc:460] Webserver started at http://127.14.16.67:42901/ using document root <none> and password file <none>
I20260807 19:42:20.212592 32522 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:20.212742 32522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:20.213063 32522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:20.216012 32522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "be930d2835dd453ca80ea9f898f01e43"
format_stamp: "Formatted at 2026-08-07 19:42:20 on dist-test-slave-07c2"
server_key: "1fd71790ea194cfcc836314b75b306db"
server_key_iv: "0eae07d41d7562b92380d1b62dc25f3d"
server_key_version: "encryptionkey@0"
I20260807 19:42:20.216792 32522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "be930d2835dd453ca80ea9f898f01e43"
format_stamp: "Formatted at 2026-08-07 19:42:20 on dist-test-slave-07c2"
server_key: "1fd71790ea194cfcc836314b75b306db"
server_key_iv: "0eae07d41d7562b92380d1b62dc25f3d"
server_key_version: "encryptionkey@0"
I20260807 19:42:20.222054 32522 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 19:42:20.225519 32537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:20.226831 32522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:42:20.227010 32522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "be930d2835dd453ca80ea9f898f01e43"
format_stamp: "Formatted at 2026-08-07 19:42:20 on dist-test-slave-07c2"
server_key: "1fd71790ea194cfcc836314b75b306db"
server_key_iv: "0eae07d41d7562b92380d1b62dc25f3d"
server_key_version: "encryptionkey@0"
I20260807 19:42:20.227226 32522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:20.253813 32522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:20.258667 32522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:20.258978 32522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:20.260185 32522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:20.261765 32522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:20.261840 32522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:20.261974 32522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:20.262048 32522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:20.297026 32522 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:45111
I20260807 19:42:20.297086 32651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:45111 every 8 connection(s)
I20260807 19:42:20.298345 32522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:20.303526 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 32522
I20260807 19:42:20.303727 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:20.304199 14401 external_mini_cluster.cc:1428] Setting key 35fd3dbac03366d6e21c1b615f992cf1
Aug 07 19:42:20 dist-test-slave-07c2 krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131740, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:42:20.318372 32654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.301009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:51647 (local address 127.14.16.126:39057)
0807 19:42:20.301478 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.301483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.302378 (+ 895us) server_negotiation.cc:407] Connection header received
0807 19:42:20.304438 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.304444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.304539 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.304659 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:20.306554 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.307618 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.309108 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.309506 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.309604 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.312876 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:20.312881 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:20.312884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:20.312918 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:20.315554 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.316209 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.316213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.316215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.316332 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.316693 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.316696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.316698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.316951 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:20.317067 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.317757 (+ 690us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.317998 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":227,"threads_started":1}
I20260807 19:42:20.319437 32655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.301517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:39057 (local address 127.14.16.67:51647)
0807 19:42:20.302195 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.302237 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.304088 (+ 1851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.304932 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.304944 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.305663 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:20.306381 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.306401 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.307765 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.307769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.308976 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.308992 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.309688 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.309693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.309881 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.310492 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:20.310513 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:20.312685 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:20.315676 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:20.315682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:20.315695 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:20.316100 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:20.316426 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:20.316429 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:20.316431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:20.316600 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:20.317056 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:20.317061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:20.317369 (+ 308us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.317619 (+ 250us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.318005 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":269,"threads_started":1}
I20260807 19:42:20.320791 32652 heartbeater.cc:347] Connected to a master server at 127.14.16.126:39057
I20260807 19:42:20.321131 32652 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:20.321851 32652 heartbeater.cc:511] Master 127.14.16.126:39057 requested a full tablet report, sending...
I20260807 19:42:20.323206 32183 ts_manager.cc:194] Registered new tserver with Master: be930d2835dd453ca80ea9f898f01e43 (127.14.16.67:45111)
I20260807 19:42:20.324033 32183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:51647
I20260807 19:42:20.332552 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:42:20.349779 32654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.337162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35472 (local address 127.14.16.126:39057)
0807 19:42:20.337409 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.337418 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.337816 (+ 398us) server_negotiation.cc:407] Connection header received
0807 19:42:20.337947 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.337951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.338019 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.338108 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:20.339610 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.340379 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.341648 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.342095 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.342195 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.344689 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:20.344694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:20.344697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:20.344727 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:20.347190 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.347877 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.347881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.347884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.348019 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.348366 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.348369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.348378 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.348736 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:20.348915 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.349222 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.349389 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:42:20.353874 32183 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:20.361346 32183 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:20.400723 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.373909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:35947 (local address 127.0.0.1:43284)
0807 19:42:20.391699 (+ 17790us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.391713 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.391862 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.392934 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.392942 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.392970 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:20.393460 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.393479 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.395315 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.395321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.397906 (+ 2585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.397923 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.399652 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.399658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.399856 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.399928 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.400072 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.400252 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17688,"mutex_wait_us":3,"thread_start_us":18109,"threads_started":1}
I20260807 19:42:20.400882 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.374188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.14.16.66:35947)
0807 19:42:20.389259 (+ 15071us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.389268 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.391831 (+ 2563us) server_negotiation.cc:407] Connection header received
0807 19:42:20.392087 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.392098 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.392367 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.392568 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:20.393677 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.395154 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.398071 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.399457 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.399608 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.399805 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.400306 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.400492 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14903,"thread_start_us":15059,"threads_started":1}
I20260807 19:42:20.400795 32667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.376529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:45111 (local address 127.0.0.1:40916)
0807 19:42:20.392933 (+ 16404us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.392958 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.393153 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.393930 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.393937 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.393963 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:20.394321 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.394330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.396401 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.396406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.398429 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.398439 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.400052 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.400060 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.400229 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.400266 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.400349 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.400479 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15649,"thread_start_us":14950,"threads_started":1}
I20260807 19:42:20.401916 32666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.373692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:37147 (local address 127.0.0.1:47464)
0807 19:42:20.388056 (+ 14364us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.388117 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.388387 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.394528 (+ 6141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.394532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.394560 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:20.394945 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.394961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.396660 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.396664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.398919 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.398936 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.401035 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.401042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.401237 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.401271 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.401380 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.401516 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13986,"mutex_wait_us":1,"thread_start_us":14945,"threads_started":1}
I20260807 19:42:20.402483 32671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.376691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40916 (local address 127.14.16.67:45111)
0807 19:42:20.392759 (+ 16068us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.392770 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.393304 (+ 534us) server_negotiation.cc:407] Connection header received
0807 19:42:20.393427 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.393437 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.393687 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.393867 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:20.394504 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.396219 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.398576 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.399830 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.399967 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.400183 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.401934 (+ 1751us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.402125 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15867,"thread_start_us":15698,"threads_started":1}
I20260807 19:42:20.402597 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.374014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47464 (local address 127.14.16.65:37147)
0807 19:42:20.393986 (+ 19972us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.393994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.394022 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:42:20.394125 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.394134 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.394332 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.394524 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:20.395133 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.396478 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.399088 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.400549 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.400682 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.400891 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.402210 (+ 1319us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.402338 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19797,"thread_start_us":19629,"threads_started":1}
I20260807 19:42:20.404911 32450 tablet_service.cc:1467] Processing CreateTablet for tablet ac8359c41a964974803110bc5ed15158 (DEFAULT_TABLE table=test-table [id=f4a2036a0fdb4884921e1a8195d9f7c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:20.405480 32585 tablet_service.cc:1467] Processing CreateTablet for tablet ac8359c41a964974803110bc5ed15158 (DEFAULT_TABLE table=test-table [id=f4a2036a0fdb4884921e1a8195d9f7c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:20.406340 32450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac8359c41a964974803110bc5ed15158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:20.406880 32585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac8359c41a964974803110bc5ed15158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:20.407968 32312 tablet_service.cc:1467] Processing CreateTablet for tablet ac8359c41a964974803110bc5ed15158 (DEFAULT_TABLE table=test-table [id=f4a2036a0fdb4884921e1a8195d9f7c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:20.409327 32312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac8359c41a964974803110bc5ed15158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:20.417124 32672 tablet_bootstrap.cc:492] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff: Bootstrap starting.
I20260807 19:42:20.421149 32673 tablet_bootstrap.cc:492] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43: Bootstrap starting.
I20260807 19:42:20.421236 32672 tablet_bootstrap.cc:654] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:20.422677 32672 log.cc:824] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:20.423143 32674 tablet_bootstrap.cc:492] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb: Bootstrap starting.
I20260807 19:42:20.424561 32673 tablet_bootstrap.cc:654] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:20.425794 32673 log.cc:824] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:20.426189 32674 tablet_bootstrap.cc:654] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:20.426447 32672 tablet_bootstrap.cc:492] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff: No bootstrap required, opened a new log
I20260807 19:42:20.426736 32672 ts_tablet_manager.cc:1397] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260807 19:42:20.427462 32674 log.cc:824] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:20.428845 32673 tablet_bootstrap.cc:492] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43: No bootstrap required, opened a new log
I20260807 19:42:20.429103 32673 ts_tablet_manager.cc:1397] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:42:20.430014 32674 tablet_bootstrap.cc:492] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb: No bootstrap required, opened a new log
I20260807 19:42:20.430337 32674 ts_tablet_manager.cc:1397] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260807 19:42:20.438267 32674 raft_consensus.cc:348] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61244b5444c446b589dee698cf1601cb" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37147 } } peers { permanent_uuid: "332dd4d510604ca5a6f7aae4447610ff" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35947 } } peers { permanent_uuid: "be930d2835dd453ca80ea9f898f01e43" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 45111 } }
I20260807 19:42:20.438683 32674 raft_consensus.cc:374] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:20.438818 32674 raft_consensus.cc:729] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61244b5444c446b589dee698cf1601cb, State: Initialized, Role: FOLLOWER
I20260807 19:42:20.438203 32672 raft_consensus.cc:348] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61244b5444c446b589dee698cf1601cb" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37147 } } peers { permanent_uuid: "332dd4d510604ca5a6f7aae4447610ff" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35947 } } peers { permanent_uuid: "be930d2835dd453ca80ea9f898f01e43" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 45111 } }
I20260807 19:42:20.439052 32672 raft_consensus.cc:374] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:20.439215 32672 raft_consensus.cc:729] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 332dd4d510604ca5a6f7aae4447610ff, State: Initialized, Role: FOLLOWER
I20260807 19:42:20.439700 32674 consensus_queue.cc:260] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61244b5444c446b589dee698cf1601cb" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37147 } } peers { permanent_uuid: "332dd4d510604ca5a6f7aae4447610ff" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35947 } } peers { permanent_uuid: "be930d2835dd453ca80ea9f898f01e43" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 45111 } }
I20260807 19:42:20.439554 32673 raft_consensus.cc:348] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61244b5444c446b589dee698cf1601cb" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37147 } } peers { permanent_uuid: "332dd4d510604ca5a6f7aae4447610ff" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35947 } } peers { permanent_uuid: "be930d2835dd453ca80ea9f898f01e43" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 45111 } }
I20260807 19:42:20.439973 32673 raft_consensus.cc:374] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:20.439848 32672 consensus_queue.cc:260] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61244b5444c446b589dee698cf1601cb" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37147 } } peers { permanent_uuid: "332dd4d510604ca5a6f7aae4447610ff" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35947 } } peers { permanent_uuid: "be930d2835dd453ca80ea9f898f01e43" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 45111 } }
I20260807 19:42:20.440071 32673 raft_consensus.cc:729] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be930d2835dd453ca80ea9f898f01e43, State: Initialized, Role: FOLLOWER
I20260807 19:42:20.440701 32673 consensus_queue.cc:260] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61244b5444c446b589dee698cf1601cb" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37147 } } peers { permanent_uuid: "332dd4d510604ca5a6f7aae4447610ff" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35947 } } peers { permanent_uuid: "be930d2835dd453ca80ea9f898f01e43" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 45111 } }
I20260807 19:42:20.442054 32652 heartbeater.cc:503] Master 127.14.16.126:39057 was elected leader, sending a full tablet report...
I20260807 19:42:20.442878 32673 ts_tablet_manager.cc:1428] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 19:42:20.447551 32674 ts_tablet_manager.cc:1428] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb: Time spent starting tablet: real 0.017s user 0.012s sys 0.005s
I20260807 19:42:20.447954 32516 heartbeater.cc:503] Master 127.14.16.126:39057 was elected leader, sending a full tablet report...
I20260807 19:42:20.449714 32672 ts_tablet_manager.cc:1428] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff: Time spent starting tablet: real 0.023s user 0.011s sys 0.010s
I20260807 19:42:20.461107 32679 raft_consensus.cc:482] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:20.461352 32679 raft_consensus.cc:504] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61244b5444c446b589dee698cf1601cb" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37147 } } peers { permanent_uuid: "332dd4d510604ca5a6f7aae4447610ff" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35947 } } peers { permanent_uuid: "be930d2835dd453ca80ea9f898f01e43" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 45111 } }
I20260807 19:42:20.462896 32679 leader_election.cc:288] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61244b5444c446b589dee698cf1601cb (127.14.16.65:37147), be930d2835dd453ca80ea9f898f01e43 (127.14.16.67:45111)
I20260807 19:42:20.470769 32671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.463428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:42863 (local address 127.14.16.67:45111)
0807 19:42:20.463613 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.463621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.464298 (+ 677us) server_negotiation.cc:407] Connection header received
0807 19:42:20.464494 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.464500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.464598 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.464693 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:20.465553 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.466731 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.468998 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.469769 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.469888 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.470002 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.470352 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.470491 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:42:20.470775 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.463278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:53425 (local address 127.14.16.65:37147)
0807 19:42:20.463517 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.463523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.464298 (+ 775us) server_negotiation.cc:407] Connection header received
0807 19:42:20.464494 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.464500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.464577 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.464684 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:20.465553 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.466756 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.469039 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.469808 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.469999 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.470073 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.470417 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.470556 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:42:20.470894 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.463241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:45111 (local address 127.14.16.66:42863)
0807 19:42:20.464188 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.464206 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.464364 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.464711 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.464716 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.464749 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:20.465409 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.465418 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.466879 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.466885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.468836 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.468848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.469989 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.469995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.470175 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.470213 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.470287 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.470416 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"mutex_wait_us":48,"spinlock_wait_cycles":33792,"thread_start_us":556,"threads_started":1}
I20260807 19:42:20.470942 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.463145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:37147 (local address 127.14.16.66:53425)
0807 19:42:20.464150 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.464171 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.464335 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.464711 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.464716 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.464764 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:20.465388 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.465407 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.466934 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.466944 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.468857 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.468870 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.470073 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.470078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.470235 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.470258 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.470376 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.470494 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":134,"spinlock_wait_cycles":4352,"thread_start_us":550,"threads_started":1}
I20260807 19:42:20.472055 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac8359c41a964974803110bc5ed15158" candidate_uuid: "332dd4d510604ca5a6f7aae4447610ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61244b5444c446b589dee698cf1601cb" is_pre_election: true
I20260807 19:42:20.472173 32606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac8359c41a964974803110bc5ed15158" candidate_uuid: "332dd4d510604ca5a6f7aae4447610ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be930d2835dd453ca80ea9f898f01e43" is_pre_election: true
I20260807 19:42:20.472609 32332 raft_consensus.cc:2454] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 332dd4d510604ca5a6f7aae4447610ff in term 0.
I20260807 19:42:20.472771 32606 raft_consensus.cc:2454] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 332dd4d510604ca5a6f7aae4447610ff in term 0.
I20260807 19:42:20.473277 32401 leader_election.cc:302] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 332dd4d510604ca5a6f7aae4447610ff, 61244b5444c446b589dee698cf1601cb; no voters:
I20260807 19:42:20.473636 32679 raft_consensus.cc:2781] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:20.473784 32679 raft_consensus.cc:482] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:20.473898 32679 raft_consensus.cc:3037] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:20.476166 32679 raft_consensus.cc:504] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61244b5444c446b589dee698cf1601cb" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37147 } } peers { permanent_uuid: "332dd4d510604ca5a6f7aae4447610ff" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35947 } } peers { permanent_uuid: "be930d2835dd453ca80ea9f898f01e43" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 45111 } }
I20260807 19:42:20.476828 32679 leader_election.cc:288] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [CANDIDATE]: Term 1 election: Requested vote from peers 61244b5444c446b589dee698cf1601cb (127.14.16.65:37147), be930d2835dd453ca80ea9f898f01e43 (127.14.16.67:45111)
I20260807 19:42:20.477200 32332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac8359c41a964974803110bc5ed15158" candidate_uuid: "332dd4d510604ca5a6f7aae4447610ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61244b5444c446b589dee698cf1601cb"
I20260807 19:42:20.477329 32606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac8359c41a964974803110bc5ed15158" candidate_uuid: "332dd4d510604ca5a6f7aae4447610ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be930d2835dd453ca80ea9f898f01e43"
I20260807 19:42:20.477398 32332 raft_consensus.cc:3037] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:20.477555 32606 raft_consensus.cc:3037] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:20.479723 32332 raft_consensus.cc:2454] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 332dd4d510604ca5a6f7aae4447610ff in term 1.
I20260807 19:42:20.479900 32606 raft_consensus.cc:2454] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 332dd4d510604ca5a6f7aae4447610ff in term 1.
I20260807 19:42:20.480180 32401 leader_election.cc:302] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 332dd4d510604ca5a6f7aae4447610ff, 61244b5444c446b589dee698cf1601cb; no voters:
I20260807 19:42:20.480454 32679 raft_consensus.cc:2781] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:20.480904 32679 raft_consensus.cc:686] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 1 LEADER]: Becoming Leader. State: Replica: 332dd4d510604ca5a6f7aae4447610ff, State: Running, Role: LEADER
I20260807 19:42:20.481346 32679 consensus_queue.cc:237] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61244b5444c446b589dee698cf1601cb" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37147 } } peers { permanent_uuid: "332dd4d510604ca5a6f7aae4447610ff" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35947 } } peers { permanent_uuid: "be930d2835dd453ca80ea9f898f01e43" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 45111 } }
I20260807 19:42:20.485726 32183 catalog_manager.cc:5495] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 332dd4d510604ca5a6f7aae4447610ff (127.14.16.66). New cstate: current_term: 1 leader_uuid: "332dd4d510604ca5a6f7aae4447610ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61244b5444c446b589dee698cf1601cb" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "332dd4d510604ca5a6f7aae4447610ff" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be930d2835dd453ca80ea9f898f01e43" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 45111 } health_report { overall_health: UNKNOWN } } }
W20260807 19:42:20.494325 32517 tablet.cc:2369] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:20.525758 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.519406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.14.16.66:35947)
0807 19:42:20.519625 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.519632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.519686 (+ 54us) server_negotiation.cc:407] Connection header received
0807 19:42:20.519910 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.519916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.520025 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.520198 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:20.520560 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.521687 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.523224 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.523914 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.524033 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.524507 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:20.525180 (+ 673us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:20.525318 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.525372 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.525458 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:42:20.534600 32332 raft_consensus.cc:1261] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb [term 1 FOLLOWER]: Refusing update from remote peer 332dd4d510604ca5a6f7aae4447610ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:20.534673 32606 raft_consensus.cc:1261] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43 [term 1 FOLLOWER]: Refusing update from remote peer 332dd4d510604ca5a6f7aae4447610ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:20.535713 32683 consensus_queue.cc:1035] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "61244b5444c446b589dee698cf1601cb" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 37147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:20.536077 32679 consensus_queue.cc:1035] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "be930d2835dd453ca80ea9f898f01e43" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:20.549297 32687 mvcc.cc:205] Tried to move back new op lower bound from 7315995609220341760 to 7315995609015697408. Current Snapshot: MvccSnapshot[applied={T|T < 7315995609220341760}]
W20260807 19:42:20.550899 32653 tablet.cc:2369] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:20.555867 32689 mvcc.cc:205] Tried to move back new op lower bound from 7315995609220341760 to 7315995609015697408. Current Snapshot: MvccSnapshot[applied={T|T < 7315995609220341760}]
I20260807 19:42:20.563010 32183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:20.563279 32183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:20.567158 32183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 09eaa5a0da9c4836ba793ff3c10a729e: Sending DeleteTablet for 3 replicas of tablet ac8359c41a964974803110bc5ed15158
I20260807 19:42:20.568498 32450 tablet_service.cc:1514] Processing DeleteTablet for tablet ac8359c41a964974803110bc5ed15158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:20 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:43284
I20260807 19:42:20.568503 32585 tablet_service.cc:1514] Processing DeleteTablet for tablet ac8359c41a964974803110bc5ed15158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:20 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:40916
I20260807 19:42:20.568881 32312 tablet_service.cc:1514] Processing DeleteTablet for tablet ac8359c41a964974803110bc5ed15158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:20 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:47464
I20260807 19:42:20.571168 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 32247
I20260807 19:42:20.580837 32695 tablet_replica.cc:330] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff: stopping tablet replica
I20260807 19:42:20.581498 32695 raft_consensus.cc:2229] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:20.582401 32695 raft_consensus.cc:2258] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:20.582648 32697 tablet_replica.cc:330] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb: stopping tablet replica
I20260807 19:42:20.583702 32697 raft_consensus.cc:2229] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:20.584810 32697 raft_consensus.cc:2258] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:20.585043 32695 ts_tablet_manager.cc:1905] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:20.587965 32697 ts_tablet_manager.cc:1905] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 19:42:20 dist-test-slave-07c2 krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:42:20.593807 32695 ts_tablet_manager.cc:1918] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:20.594470 32696 tablet_replica.cc:330] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43: stopping tablet replica
I20260807 19:42:20.595000 32696 raft_consensus.cc:2229] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:20.595639 32696 raft_consensus.cc:2258] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:20.597587 32696 ts_tablet_manager.cc:1905] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:20.598979 32697 ts_tablet_manager.cc:1918] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:20.599133 32697 log.cc:1196] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/ac8359c41a964974803110bc5ed15158
I20260807 19:42:20.599725 32697 ts_tablet_manager.cc:1939] T ac8359c41a964974803110bc5ed15158 P 61244b5444c446b589dee698cf1601cb: Deleting consensus metadata
I20260807 19:42:20.601205 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.574867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47478 (local address 127.14.16.65:37147)
0807 19:42:20.575061 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.575068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.578748 (+ 3680us) server_negotiation.cc:407] Connection header received
0807 19:42:20.579816 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.579825 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.579937 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.580029 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:20.582713 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.583895 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.585891 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.586568 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.586691 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.592392 (+ 5701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:20.592407 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:20.592426 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:20.592478 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:20.597269 (+ 4791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.598249 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.598260 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.598264 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.598441 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.598971 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.598977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.598980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.599504 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:20.599811 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.600513 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.600758 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:42:20.601691 32167 catalog_manager.cc:4841] TS 61244b5444c446b589dee698cf1601cb (127.14.16.65:37147): tablet ac8359c41a964974803110bc5ed15158 (table test-table [id=f4a2036a0fdb4884921e1a8195d9f7c4]) successfully deleted
I20260807 19:42:20.596180 32695 log.cc:1196] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/ac8359c41a964974803110bc5ed15158
I20260807 19:42:20.603394 32374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:20.603590 32695 ts_tablet_manager.cc:1939] T ac8359c41a964974803110bc5ed15158 P 332dd4d510604ca5a6f7aae4447610ff: Deleting consensus metadata
I20260807 19:42:20.604278 32696 ts_tablet_manager.cc:1918] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:20.604456 32696 log.cc:1196] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/ac8359c41a964974803110bc5ed15158
I20260807 19:42:20.604992 32696 ts_tablet_manager.cc:1939] T ac8359c41a964974803110bc5ed15158 P be930d2835dd453ca80ea9f898f01e43: Deleting consensus metadata
I20260807 19:42:20.605979 32168 catalog_manager.cc:4841] TS 332dd4d510604ca5a6f7aae4447610ff (127.14.16.66:35947): tablet ac8359c41a964974803110bc5ed15158 (table test-table [id=f4a2036a0fdb4884921e1a8195d9f7c4]) successfully deleted
I20260807 19:42:20.606659 32170 catalog_manager.cc:4841] TS be930d2835dd453ca80ea9f898f01e43 (127.14.16.67:45111): tablet ac8359c41a964974803110bc5ed15158 (table test-table [id=f4a2036a0fdb4884921e1a8195d9f7c4]) successfully deleted
I20260807 19:42:20.696879 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 32247
I20260807 19:42:20.716856 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 32385
Aug 07 19:42:20 dist-test-slave-07c2 krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:42:20.733369 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.717796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43292 (local address 127.14.16.66:35947)
0807 19:42:20.718007 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.718015 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.718040 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:42:20.718129 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.718134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.718240 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.718356 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:20.719748 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.720602 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.721906 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.722384 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.722518 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.726871 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:20.726886 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:20.726901 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:20.726941 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:20.730099 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.730853 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.730868 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.730873 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.731040 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.731721 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.731726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.731731 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.732175 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:20.732501 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.732771 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.733000 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:42:20.734097 32510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:20.848403 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 32385
I20260807 19:42:20.870146 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 32522
Aug 07 19:42:20 dist-test-slave-07c2 krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:42:20.885934 32671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.870874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40932 (local address 127.14.16.67:45111)
0807 19:42:20.871061 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.871070 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.871151 (+ 81us) server_negotiation.cc:407] Connection header received
0807 19:42:20.871411 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.871417 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.871505 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.871614 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:20.873033 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.873783 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.875114 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.875633 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.875778 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.880159 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:20.880173 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:20.880190 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:20.880224 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:20.883033 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.883813 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.883820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.883823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.883966 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.884363 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.884368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.884370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.884721 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:20.885042 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.885370 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.885613 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:42:20.886587 32647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:20.977437 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 32522
I20260807 19:42:20.996524 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 32150
I20260807 19:42:20.997676 32214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:21.110472 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 32150
2026-08-07T19:42:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4536 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T19:42:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:21Z Disabled control of system clock
I20260807 19:42:21.204607 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:40567
--webserver_interface=127.14.16.126
--webserver_port=0
--builtin_ntp_servers=127.14.16.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:40567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:42:21.558780 32706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:21.559083 32706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:21.559170 32706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:21.566828 32706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:21.566946 32706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:21.566984 32706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:21.567034 32706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:21.567080 32706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:21.580312 32706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:40567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:40567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:21.581612 32706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:21.583505 32706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:21.595386 32711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:21.595415 32712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:21.595335 32714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:21.596347 32706 server_base.cc:1034] running on GCE node
I20260807 19:42:21.597041 32706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:21.598428 32706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:21.599622 32706 hybrid_clock.cc:648] HybridClock initialized: now 1786131741599558 us; error 55 us; skew 500 ppm
I20260807 19:42:21.599972 32706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:21.606906 32706 webserver.cc:460] Webserver started at http://127.14.16.126:35765/ using document root <none> and password file <none>
I20260807 19:42:21.607707 32706 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:21.607831 32706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:21.608175 32706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:21.611104 32706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "d6153ea49b9b4ba7a3c38cb153daa20c"
format_stamp: "Formatted at 2026-08-07 19:42:21 on dist-test-slave-07c2"
server_key: "2e6febfa1e35473c4beab6775ea74217"
server_key_iv: "830c3be3288589e1c5ff17fc2aa29f20"
server_key_version: "encryptionkey@0"
I20260807 19:42:21.611981 32706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "d6153ea49b9b4ba7a3c38cb153daa20c"
format_stamp: "Formatted at 2026-08-07 19:42:21 on dist-test-slave-07c2"
server_key: "2e6febfa1e35473c4beab6775ea74217"
server_key_iv: "830c3be3288589e1c5ff17fc2aa29f20"
server_key_version: "encryptionkey@0"
I20260807 19:42:21.617002 32706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:21.620345 32720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:21.621685 32706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:42:21.621879 32706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "d6153ea49b9b4ba7a3c38cb153daa20c"
format_stamp: "Formatted at 2026-08-07 19:42:21 on dist-test-slave-07c2"
server_key: "2e6febfa1e35473c4beab6775ea74217"
server_key_iv: "830c3be3288589e1c5ff17fc2aa29f20"
server_key_version: "encryptionkey@0"
I20260807 19:42:21.622098 32706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:21.635705 32706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:21.636482 32706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:21.636765 32706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:21.655025 32706 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:40567
I20260807 19:42:21.655031 304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:40567 every 8 connection(s)
I20260807 19:42:21.656406 32706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:21.660207 305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:21.663786 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 32706
I20260807 19:42:21.663972 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:21.664433 14401 external_mini_cluster.cc:1428] Setting key 0445c1d0341f6d1661c09c5d748d683d
I20260807 19:42:21.673928 305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c: Bootstrap starting.
I20260807 19:42:21.677960 305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:21.679419 305 log.cc:824] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:21.679536 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:21.666288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.14.16.126:40567)
0807 19:42:21.667401 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:21.667415 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:21.667468 (+ 53us) server_negotiation.cc:407] Connection header received
0807 19:42:21.669372 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:21.669404 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:21.670009 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:21.670450 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:21.670992 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:21.673015 (+ 2023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:21.674402 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:21.675449 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:21.675670 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:21.676760 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:21.676771 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:21.676787 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:21.677491 (+ 704us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:21.677610 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:21.677647 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:21.677755 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:21.677959 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 19:42:21.678258 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":345,"threads_started":1}
I20260807 19:42:21.682847 305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c: No bootstrap required, opened a new log
I20260807 19:42:21.690436 305 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6153ea49b9b4ba7a3c38cb153daa20c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40567 } }
I20260807 19:42:21.690788 305 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:21.690867 305 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6153ea49b9b4ba7a3c38cb153daa20c, State: Initialized, Role: FOLLOWER
I20260807 19:42:21.691485 305 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6153ea49b9b4ba7a3c38cb153daa20c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40567 } }
I20260807 19:42:21.691707 305 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:21.691807 305 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:21.691931 305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:21.694419 305 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6153ea49b9b4ba7a3c38cb153daa20c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40567 } }
I20260807 19:42:21.694898 305 leader_election.cc:302] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6153ea49b9b4ba7a3c38cb153daa20c; no voters:
I20260807 19:42:21.695611 305 leader_election.cc:288] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:21.695772 310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:21.696498 310 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [term 1 LEADER]: Becoming Leader. State: Replica: d6153ea49b9b4ba7a3c38cb153daa20c, State: Running, Role: LEADER
I20260807 19:42:21.697046 310 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6153ea49b9b4ba7a3c38cb153daa20c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40567 } }
I20260807 19:42:21.697633 305 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:21.700098 312 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6153ea49b9b4ba7a3c38cb153daa20c. Latest consensus state: current_term: 1 leader_uuid: "d6153ea49b9b4ba7a3c38cb153daa20c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6153ea49b9b4ba7a3c38cb153daa20c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40567 } } }
I20260807 19:42:21.700178 311 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6153ea49b9b4ba7a3c38cb153daa20c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6153ea49b9b4ba7a3c38cb153daa20c" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 40567 } } }
I20260807 19:42:21.700536 312 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:21.700613 311 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:21.702230 318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:21.707996 318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:21.716961 318 catalog_manager.cc:1347] Generated new cluster ID: 2f02746cc8dc47069a65b55fe9665675
I20260807 19:42:21.717103 318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:21.741303 318 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:21.743084 318 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:21.753804 318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c: Generated new TSK 0
I20260807 19:42:21.754889 318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:21.762796 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40567
--builtin_ntp_servers=127.14.16.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:42:22.108863 329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:22.109153 329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:22.109242 329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:22.116472 329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:22.116586 329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:22.116696 329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:42:22.126206 329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:22.127583 329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:22.129307 329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:22.142442 335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:22.142488 336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:22.142513 338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:22.143234 329 server_base.cc:1034] running on GCE node
I20260807 19:42:22.143893 329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:22.144834 329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:22.146040 329 hybrid_clock.cc:648] HybridClock initialized: now 1786131742146000 us; error 29 us; skew 500 ppm
I20260807 19:42:22.146353 329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:22.148581 329 webserver.cc:460] Webserver started at http://127.14.16.65:32919/ using document root <none> and password file <none>
I20260807 19:42:22.149251 329 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:22.149364 329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:22.149700 329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:22.153544 329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "67e9984475634f5eb25e793c3c9da9a2"
format_stamp: "Formatted at 2026-08-07 19:42:22 on dist-test-slave-07c2"
server_key: "0420a83d2391e98fd80d7438f0cd72c5"
server_key_iv: "b694228b9be5e517d1e4b4c08795ce14"
server_key_version: "encryptionkey@0"
I20260807 19:42:22.154433 329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "67e9984475634f5eb25e793c3c9da9a2"
format_stamp: "Formatted at 2026-08-07 19:42:22 on dist-test-slave-07c2"
server_key: "0420a83d2391e98fd80d7438f0cd72c5"
server_key_iv: "b694228b9be5e517d1e4b4c08795ce14"
server_key_version: "encryptionkey@0"
I20260807 19:42:22.159587 329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 19:42:22.162962 344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:22.164439 329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:22.164620 329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "67e9984475634f5eb25e793c3c9da9a2"
format_stamp: "Formatted at 2026-08-07 19:42:22 on dist-test-slave-07c2"
server_key: "0420a83d2391e98fd80d7438f0cd72c5"
server_key_iv: "b694228b9be5e517d1e4b4c08795ce14"
server_key_version: "encryptionkey@0"
I20260807 19:42:22.164848 329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:22.210520 329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:22.211491 329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:22.211814 329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:22.212829 329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:22.214417 329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:22.214540 329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:22.214670 329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:22.214756 329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:22.248566 329 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:39679
I20260807 19:42:22.248663 459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:39679 every 8 connection(s)
I20260807 19:42:22.249899 329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:22.255347 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 329
I20260807 19:42:22.255553 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:22.256151 14401 external_mini_cluster.cc:1428] Setting key 2e0a821709bbc3a5f2275e12dae758ef
I20260807 19:42:22.259833 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40567
--builtin_ntp_servers=127.14.16.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:42:22.265955 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:22.252551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:46567 (local address 127.14.16.126:40567)
0807 19:42:22.253215 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:22.253223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:22.254466 (+ 1243us) server_negotiation.cc:407] Connection header received
0807 19:42:22.257209 (+ 2743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:22.257215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:22.257311 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:22.257408 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:22.258979 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:22.259824 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:22.262739 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:22.263151 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:22.263257 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:22.265136 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:22.265141 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:22.265144 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:22.265164 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:22.265205 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:22.265217 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:22.265305 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:22.265603 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 19:42:22.265686 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":383,"threads_started":1}
I20260807 19:42:22.267627 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:22.253378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40567 (local address 127.14.16.65:46567)
0807 19:42:22.254307 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:22.254342 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:22.256925 (+ 2583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:22.257755 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:22.257767 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:22.257892 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:22.258777 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:22.258804 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:22.260024 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:22.260032 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:22.262515 (+ 2483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:22.262534 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:22.263425 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:22.263435 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:22.263806 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:22.264717 (+ 911us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:22.264723 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:22.264783 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:22.264898 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:22.265334 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:22.265349 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:22.265360 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:42:22.265564 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 19:42:22.266045 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":298,"threads_started":1}
I20260807 19:42:22.269423 460 heartbeater.cc:347] Connected to a master server at 127.14.16.126:40567
I20260807 19:42:22.269776 460 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:22.270759 460 heartbeater.cc:511] Master 127.14.16.126:40567 requested a full tablet report, sending...
I20260807 19:42:22.272987 32737 ts_manager.cc:194] Registered new tserver with Master: 67e9984475634f5eb25e793c3c9da9a2 (127.14.16.65:39679)
I20260807 19:42:22.275043 32737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.16.65:46567
W20260807 19:42:22.601960 464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:22.602317 464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:22.602442 464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:22.609942 464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:22.610049 464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:22.610163 464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:42:22.619422 464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:22.620677 464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:22.622510 464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:22.635812 470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:22.635907 472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:22.635864 469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:22.636534 464 server_base.cc:1034] running on GCE node
I20260807 19:42:22.637244 464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:22.638255 464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:22.639494 464 hybrid_clock.cc:648] HybridClock initialized: now 1786131742639452 us; error 56 us; skew 500 ppm
I20260807 19:42:22.639880 464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:22.642287 464 webserver.cc:460] Webserver started at http://127.14.16.66:37497/ using document root <none> and password file <none>
I20260807 19:42:22.643018 464 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:22.643160 464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:22.643541 464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:22.646497 464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "b6ce0bf86ac946d19dbd135886445059"
format_stamp: "Formatted at 2026-08-07 19:42:22 on dist-test-slave-07c2"
server_key: "255c2302bd9ff681b4587096cb0bd446"
server_key_iv: "6e596aeada86880df664c570d050b64c"
server_key_version: "encryptionkey@0"
I20260807 19:42:22.647380 464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "b6ce0bf86ac946d19dbd135886445059"
format_stamp: "Formatted at 2026-08-07 19:42:22 on dist-test-slave-07c2"
server_key: "255c2302bd9ff681b4587096cb0bd446"
server_key_iv: "6e596aeada86880df664c570d050b64c"
server_key_version: "encryptionkey@0"
I20260807 19:42:22.652562 464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:22.656244 478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:22.657827 464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 19:42:22.657986 464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "b6ce0bf86ac946d19dbd135886445059"
format_stamp: "Formatted at 2026-08-07 19:42:22 on dist-test-slave-07c2"
server_key: "255c2302bd9ff681b4587096cb0bd446"
server_key_iv: "6e596aeada86880df664c570d050b64c"
server_key_version: "encryptionkey@0"
I20260807 19:42:22.658200 464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:22.676777 464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:22.677490 464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:22.677747 464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:22.678725 464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:22.680197 464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:22.680284 464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:22.680357 464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:22.680413 464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:22.712340 464 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:44743
I20260807 19:42:22.712419 594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:44743 every 8 connection(s)
I20260807 19:42:22.713733 464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:22.723708 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 464
I20260807 19:42:22.723932 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:22.724534 14401 external_mini_cluster.cc:1428] Setting key 0f76092897b5dcab9e725abce121fe6c
I20260807 19:42:22.726339 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:22.716405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:37993 (local address 127.14.16.126:40567)
0807 19:42:22.716636 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:22.716643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:22.717920 (+ 1277us) server_negotiation.cc:407] Connection header received
0807 19:42:22.719707 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:22.719712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:22.719782 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:22.719872 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:22.720867 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:22.721618 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:22.723112 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:22.723610 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:22.723715 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:22.725385 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:22.725390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:22.725394 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:22.725432 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:22.725475 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:22.725493 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:22.725612 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:22.725936 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 19:42:22.726017 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:42:22.727496 597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:22.716888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40567 (local address 127.14.16.66:37993)
0807 19:42:22.717752 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:22.717791 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:22.719452 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:22.720065 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:22.720073 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:22.720153 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:22.720714 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:22.720730 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:22.721746 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:22.721751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:22.722967 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:22.722978 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:22.723867 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:22.723876 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:22.724181 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:22.724974 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:22.724979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:22.725034 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:22.725134 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:22.725676 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:22.725686 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:22.725710 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:42:22.725920 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 19:42:22.726341 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":290,"threads_started":1}
I20260807 19:42:22.727804 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40567
--builtin_ntp_servers=127.14.16.84:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:42:22.728969 595 heartbeater.cc:347] Connected to a master server at 127.14.16.126:40567
I20260807 19:42:22.729280 595 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:22.730021 595 heartbeater.cc:511] Master 127.14.16.126:40567 requested a full tablet report, sending...
I20260807 19:42:22.731470 32737 ts_manager.cc:194] Registered new tserver with Master: b6ce0bf86ac946d19dbd135886445059 (127.14.16.66:44743)
I20260807 19:42:22.732272 32737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.16.66:37993
W20260807 19:42:23.060120 598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:23.060396 598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:23.060655 598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:23.068105 598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:23.068204 598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:23.068400 598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:42:23.077389 598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:40567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:23.078691 598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:23.080452 598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:23.093660 606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:23.093695 603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:23.093600 604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:23.094581 598 server_base.cc:1034] running on GCE node
I20260807 19:42:23.095155 598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:23.096112 598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:23.097364 598 hybrid_clock.cc:648] HybridClock initialized: now 1786131743097318 us; error 31 us; skew 500 ppm
I20260807 19:42:23.097703 598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:23.099999 598 webserver.cc:460] Webserver started at http://127.14.16.67:41727/ using document root <none> and password file <none>
I20260807 19:42:23.100658 598 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:23.100780 598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:23.101109 598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:23.104059 598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "f7a13548bd2d41d7a9a6b575a8d134e8"
format_stamp: "Formatted at 2026-08-07 19:42:23 on dist-test-slave-07c2"
server_key: "e511c8fcf40e844cc4e6c02744adf5d1"
server_key_iv: "167ecf21facb103ed5b4ed78de0998b7"
server_key_version: "encryptionkey@0"
I20260807 19:42:23.104830 598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "f7a13548bd2d41d7a9a6b575a8d134e8"
format_stamp: "Formatted at 2026-08-07 19:42:23 on dist-test-slave-07c2"
server_key: "e511c8fcf40e844cc4e6c02744adf5d1"
server_key_iv: "167ecf21facb103ed5b4ed78de0998b7"
server_key_version: "encryptionkey@0"
I20260807 19:42:23.109737 598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:42:23.113185 612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:23.114470 598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:42:23.114637 598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "f7a13548bd2d41d7a9a6b575a8d134e8"
format_stamp: "Formatted at 2026-08-07 19:42:23 on dist-test-slave-07c2"
server_key: "e511c8fcf40e844cc4e6c02744adf5d1"
server_key_iv: "167ecf21facb103ed5b4ed78de0998b7"
server_key_version: "encryptionkey@0"
I20260807 19:42:23.114853 598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:23.141554 598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:23.142472 598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:23.142827 598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:23.143972 598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:23.145539 598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:23.145679 598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:23.145816 598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:23.145920 598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:23.183522 598 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:40613
I20260807 19:42:23.183578 726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:40613 every 8 connection(s)
I20260807 19:42:23.185007 598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:23.189247 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 598
I20260807 19:42:23.189438 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:23.190030 14401 external_mini_cluster.cc:1428] Setting key cf3be2d6de24ae66eeccea0d6e87dffb
I20260807 19:42:23.198371 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.187825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:53921 (local address 127.14.16.126:40567)
0807 19:42:23.188018 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.188023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.189721 (+ 1698us) server_negotiation.cc:407] Connection header received
0807 19:42:23.192190 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.192200 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.192289 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.192397 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.193576 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.194312 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.195892 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.196280 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.196372 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.197671 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.197676 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.197679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.197697 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.197730 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.197741 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.197815 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.198026 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.198108 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:42:23.199476 729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.188387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:40567 (local address 127.14.16.67:53921)
0807 19:42:23.189538 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.189588 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.191883 (+ 2295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.192583 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.192591 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.192679 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:23.193428 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.193449 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.194431 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.194434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.195770 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.195783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.196464 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.196469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.196677 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.197313 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:23.197316 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:23.197355 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:23.197437 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:23.197834 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:23.197843 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:23.197849 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.197998 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.198314 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":343,"threads_started":1}
I20260807 19:42:23.200860 727 heartbeater.cc:347] Connected to a master server at 127.14.16.126:40567
I20260807 19:42:23.201164 727 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:23.202014 727 heartbeater.cc:511] Master 127.14.16.126:40567 requested a full tablet report, sending...
I20260807 19:42:23.203625 32737 ts_manager.cc:194] Registered new tserver with Master: f7a13548bd2d41d7a9a6b575a8d134e8 (127.14.16.67:40613)
I20260807 19:42:23.204679 32737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.16.67:53921
I20260807 19:42:23.205629 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:42:23.215440 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.209826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41412 (local address 127.14.16.126:40567)
0807 19:42:23.210090 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.210096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.210398 (+ 302us) server_negotiation.cc:407] Connection header received
0807 19:42:23.210532 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.210536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.210604 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.210685 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.211186 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.211910 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.213174 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.213577 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.213675 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.214659 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.214664 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.214667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.214684 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.214716 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.214727 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.214799 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.214921 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.214992 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:42:23.219827 32736 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:23.227222 32736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:23.253455 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.239325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39679 (local address 127.0.0.1:46714)
0807 19:42:23.240855 (+ 1530us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.240891 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.241074 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.244998 (+ 3924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.245005 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.245041 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:23.245425 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.245444 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.247822 (+ 2378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.247830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.250504 (+ 2674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.250521 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.252495 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.252502 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.252711 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.252764 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.252869 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.253065 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1275,"mutex_wait_us":47,"thread_start_us":2033,"threads_started":1}
I20260807 19:42:23.253547 739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.239621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:40613 (local address 127.0.0.1:36846)
0807 19:42:23.241673 (+ 2052us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.241690 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.241800 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.245630 (+ 3830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.245637 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.245690 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:23.246145 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.246161 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.248437 (+ 2276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.248442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.250711 (+ 2269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.250729 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.252799 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.252805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.252984 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.253020 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.253089 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.253228 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1877,"thread_start_us":820,"threads_started":1}
I20260807 19:42:23.254532 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.240378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36846 (local address 127.14.16.67:40613)
0807 19:42:23.244973 (+ 4595us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.244983 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.245018 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:42:23.245120 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.245129 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.245371 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.246380 (+ 1009us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:23.246695 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.248274 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.250936 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.252552 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.253690 (+ 1138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.253938 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.254011 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.254184 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4414,"thread_start_us":4248,"threads_started":1}
I20260807 19:42:23.254786 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.241247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:44743 (local address 127.0.0.1:57746)
0807 19:42:23.244039 (+ 2792us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.244058 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.244190 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.245247 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.245252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.245272 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:23.245732 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.245744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.247690 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.247696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.251655 (+ 3959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.251669 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.253961 (+ 2292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.253969 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.254163 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.254205 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.254296 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.254426 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":1297,"thread_start_us":791,"threads_started":1}
I20260807 19:42:23.255594 741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.240022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.14.16.65:39679)
0807 19:42:23.244471 (+ 4449us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.244479 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.244503 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:42:23.244589 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.244596 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.244770 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.245700 (+ 930us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:23.245954 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.247662 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.250757 (+ 3095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.252268 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.254553 (+ 2285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.254908 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.254982 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.255164 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4290,"thread_start_us":4171,"threads_started":1}
I20260807 19:42:23.256266 744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.241771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57746 (local address 127.14.16.66:44743)
0807 19:42:23.244027 (+ 2256us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.244036 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.244317 (+ 281us) server_negotiation.cc:407] Connection header received
0807 19:42:23.244440 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.244448 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.244710 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.244942 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:23.245945 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.247536 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.251897 (+ 4361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.253411 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.253552 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.253817 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.255052 (+ 1235us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.255239 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2105,"thread_start_us":2011,"threads_started":1}
I20260807 19:42:23.258534 661 tablet_service.cc:1467] Processing CreateTablet for tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3 (DEFAULT_TABLE table=test-table [id=200a6d76979b4c2a915093990c92ce6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:23.259142 393 tablet_service.cc:1467] Processing CreateTablet for tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3 (DEFAULT_TABLE table=test-table [id=200a6d76979b4c2a915093990c92ce6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:23.259977 661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:23.259598 529 tablet_service.cc:1467] Processing CreateTablet for tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3 (DEFAULT_TABLE table=test-table [id=200a6d76979b4c2a915093990c92ce6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:23.260576 393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:23.260885 529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:23.272832 745 tablet_bootstrap.cc:492] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8: Bootstrap starting.
I20260807 19:42:23.275583 746 tablet_bootstrap.cc:492] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2: Bootstrap starting.
I20260807 19:42:23.276445 747 tablet_bootstrap.cc:492] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059: Bootstrap starting.
I20260807 19:42:23.276743 745 tablet_bootstrap.cc:654] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:23.278213 745 log.cc:824] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:23.278277 460 heartbeater.cc:503] Master 127.14.16.126:40567 was elected leader, sending a full tablet report...
I20260807 19:42:23.279892 746 tablet_bootstrap.cc:654] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:23.280565 747 tablet_bootstrap.cc:654] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:23.281299 746 log.cc:824] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:23.282181 747 log.cc:824] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:23.288024 747 tablet_bootstrap.cc:492] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059: No bootstrap required, opened a new log
I20260807 19:42:23.288020 746 tablet_bootstrap.cc:492] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2: No bootstrap required, opened a new log
I20260807 19:42:23.288022 745 tablet_bootstrap.cc:492] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8: No bootstrap required, opened a new log
I20260807 19:42:23.288334 747 ts_tablet_manager.cc:1397] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260807 19:42:23.288353 745 ts_tablet_manager.cc:1397] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260807 19:42:23.288358 746 ts_tablet_manager.cc:1397] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260807 19:42:23.296154 747 raft_consensus.cc:348] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e9984475634f5eb25e793c3c9da9a2" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39679 } } peers { permanent_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40613 } } peers { permanent_uuid: "b6ce0bf86ac946d19dbd135886445059" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44743 } }
I20260807 19:42:23.296607 747 raft_consensus.cc:374] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:23.296779 747 raft_consensus.cc:729] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ce0bf86ac946d19dbd135886445059, State: Initialized, Role: FOLLOWER
I20260807 19:42:23.297642 747 consensus_queue.cc:260] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e9984475634f5eb25e793c3c9da9a2" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39679 } } peers { permanent_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40613 } } peers { permanent_uuid: "b6ce0bf86ac946d19dbd135886445059" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44743 } }
I20260807 19:42:23.299106 595 heartbeater.cc:503] Master 127.14.16.126:40567 was elected leader, sending a full tablet report...
I20260807 19:42:23.299969 747 ts_tablet_manager.cc:1428] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260807 19:42:23.299839 746 raft_consensus.cc:348] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e9984475634f5eb25e793c3c9da9a2" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39679 } } peers { permanent_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40613 } } peers { permanent_uuid: "b6ce0bf86ac946d19dbd135886445059" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44743 } }
I20260807 19:42:23.299839 745 raft_consensus.cc:348] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e9984475634f5eb25e793c3c9da9a2" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39679 } } peers { permanent_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40613 } } peers { permanent_uuid: "b6ce0bf86ac946d19dbd135886445059" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44743 } }
I20260807 19:42:23.300324 746 raft_consensus.cc:374] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:23.300325 745 raft_consensus.cc:374] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:23.300433 745 raft_consensus.cc:729] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7a13548bd2d41d7a9a6b575a8d134e8, State: Initialized, Role: FOLLOWER
I20260807 19:42:23.300465 746 raft_consensus.cc:729] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67e9984475634f5eb25e793c3c9da9a2, State: Initialized, Role: FOLLOWER
I20260807 19:42:23.301108 746 consensus_queue.cc:260] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e9984475634f5eb25e793c3c9da9a2" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39679 } } peers { permanent_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40613 } } peers { permanent_uuid: "b6ce0bf86ac946d19dbd135886445059" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44743 } }
I20260807 19:42:23.301115 745 consensus_queue.cc:260] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e9984475634f5eb25e793c3c9da9a2" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39679 } } peers { permanent_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40613 } } peers { permanent_uuid: "b6ce0bf86ac946d19dbd135886445059" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44743 } }
I20260807 19:42:23.302609 727 heartbeater.cc:503] Master 127.14.16.126:40567 was elected leader, sending a full tablet report...
I20260807 19:42:23.303228 745 ts_tablet_manager.cc:1428] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260807 19:42:23.303573 746 ts_tablet_manager.cc:1428] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260807 19:42:23.313493 753 raft_consensus.cc:482] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:23.313792 753 raft_consensus.cc:504] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e9984475634f5eb25e793c3c9da9a2" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39679 } } peers { permanent_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40613 } } peers { permanent_uuid: "b6ce0bf86ac946d19dbd135886445059" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44743 } }
I20260807 19:42:23.315583 753 leader_election.cc:288] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67e9984475634f5eb25e793c3c9da9a2 (127.14.16.65:39679), b6ce0bf86ac946d19dbd135886445059 (127.14.16.66:44743)
I20260807 19:42:23.317447 752 raft_consensus.cc:482] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:23.317780 752 raft_consensus.cc:504] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e9984475634f5eb25e793c3c9da9a2" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39679 } } peers { permanent_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40613 } } peers { permanent_uuid: "b6ce0bf86ac946d19dbd135886445059" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44743 } }
I20260807 19:42:23.319702 752 leader_election.cc:288] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7a13548bd2d41d7a9a6b575a8d134e8 (127.14.16.67:40613), b6ce0bf86ac946d19dbd135886445059 (127.14.16.66:44743)
I20260807 19:42:23.323845 729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.315748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39679 (local address 127.14.16.67:60849)
0807 19:42:23.316021 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.316045 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.316181 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.316646 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.316654 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.316694 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:23.316886 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.316896 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.318416 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.318422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.320570 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.320586 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.322907 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.322917 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.323208 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.323271 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.323401 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.323542 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168}
I20260807 19:42:23.324361 744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.316228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:34495 (local address 127.14.16.66:44743)
0807 19:42:23.316418 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.316423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.316966 (+ 543us) server_negotiation.cc:407] Connection header received
0807 19:42:23.317093 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.317097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.317167 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.317270 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:23.317863 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.318924 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.321519 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.322794 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.322919 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.322981 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.323930 (+ 949us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.324075 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:42:23.324844 741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.315813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:60849 (local address 127.14.16.65:39679)
0807 19:42:23.316005 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.316011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.316103 (+ 92us) server_negotiation.cc:407] Connection header received
0807 19:42:23.316325 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.316332 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.316444 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.316566 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:23.317090 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.318278 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.320980 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.322510 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.322700 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.322800 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.324137 (+ 1337us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.324292 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:42:23.325117 754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.315969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:44743 (local address 127.14.16.67:34495)
0807 19:42:23.316856 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.316869 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.316967 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.317297 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.317301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.317324 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:23.317641 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.317654 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.319095 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.319100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.321283 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.321296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.323533 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.323539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.323719 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.323759 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.324628 (+ 869us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.324763 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":459,"threads_started":1}
I20260807 19:42:23.326310 413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0b3a5f341b4d7da5d3cbe1dc4b79a3" candidate_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67e9984475634f5eb25e793c3c9da9a2" is_pre_election: true
I20260807 19:42:23.326813 413 raft_consensus.cc:2454] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7a13548bd2d41d7a9a6b575a8d134e8 in term 0.
I20260807 19:42:23.327654 614 leader_election.cc:302] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67e9984475634f5eb25e793c3c9da9a2, f7a13548bd2d41d7a9a6b575a8d134e8; no voters:
I20260807 19:42:23.327961 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.320468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:52697 (local address 127.14.16.67:40613)
0807 19:42:23.320838 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.320845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.320869 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:42:23.321028 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.321033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.321094 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.321148 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:23.322019 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.323092 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.325640 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.326822 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.326991 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.327055 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.327571 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.327711 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247}
I20260807 19:42:23.328347 753 raft_consensus.cc:2781] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:23.328586 753 raft_consensus.cc:482] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:23.328660 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.320000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:40613 (local address 127.14.16.65:52697)
0807 19:42:23.320732 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.320752 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.320877 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.321336 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.321342 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.321373 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:23.321850 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.321863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.323265 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.323270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.325445 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.325466 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.327049 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.327058 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.327348 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.327403 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.328167 (+ 764us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.328335 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":521,"threads_started":1}
I20260807 19:42:23.328569 549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0b3a5f341b4d7da5d3cbe1dc4b79a3" candidate_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ce0bf86ac946d19dbd135886445059" is_pre_election: true
I20260807 19:42:23.328876 753 raft_consensus.cc:3037] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:23.329180 549 raft_consensus.cc:2454] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7a13548bd2d41d7a9a6b575a8d134e8 in term 0.
I20260807 19:42:23.330312 682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0b3a5f341b4d7da5d3cbe1dc4b79a3" candidate_uuid: "67e9984475634f5eb25e793c3c9da9a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" is_pre_election: true
I20260807 19:42:23.331796 753 raft_consensus.cc:504] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e9984475634f5eb25e793c3c9da9a2" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39679 } } peers { permanent_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40613 } } peers { permanent_uuid: "b6ce0bf86ac946d19dbd135886445059" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44743 } }
I20260807 19:42:23.332237 682 raft_consensus.cc:2379] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67e9984475634f5eb25e793c3c9da9a2 in current term 1: Already voted for candidate f7a13548bd2d41d7a9a6b575a8d134e8 in this term.
I20260807 19:42:23.332748 753 leader_election.cc:288] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [CANDIDATE]: Term 1 election: Requested vote from peers 67e9984475634f5eb25e793c3c9da9a2 (127.14.16.65:39679), b6ce0bf86ac946d19dbd135886445059 (127.14.16.66:44743)
I20260807 19:42:23.332998 413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0b3a5f341b4d7da5d3cbe1dc4b79a3" candidate_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67e9984475634f5eb25e793c3c9da9a2"
I20260807 19:42:23.333217 413 raft_consensus.cc:3037] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:23.334501 549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0b3a5f341b4d7da5d3cbe1dc4b79a3" candidate_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ce0bf86ac946d19dbd135886445059"
I20260807 19:42:23.334682 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.320991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:44743 (local address 127.14.16.65:39159)
0807 19:42:23.328047 (+ 7056us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.328068 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.328276 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.328879 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.328884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.328913 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:23.329277 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.329292 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.330767 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.330771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.332329 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:23.332347 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.334014 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.334021 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:23.334206 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.334244 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.334348 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.334457 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6931,"thread_start_us":6864,"threads_started":1}
I20260807 19:42:23.334723 549 raft_consensus.cc:3037] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:23.335458 744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.321337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:39159 (local address 127.14.16.66:44743)
0807 19:42:23.324581 (+ 3244us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.324587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.328167 (+ 3580us) server_negotiation.cc:407] Connection header received
0807 19:42:23.328438 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.328445 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.328588 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.328661 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:23.329442 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.330591 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.332571 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.333803 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.333964 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.334033 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.335061 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.335173 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3149,"thread_start_us":6045,"threads_started":1}
I20260807 19:42:23.336025 548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0b3a5f341b4d7da5d3cbe1dc4b79a3" candidate_uuid: "67e9984475634f5eb25e793c3c9da9a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ce0bf86ac946d19dbd135886445059" is_pre_election: true
I20260807 19:42:23.336588 413 raft_consensus.cc:2454] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7a13548bd2d41d7a9a6b575a8d134e8 in term 1.
I20260807 19:42:23.336896 347 leader_election.cc:302] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67e9984475634f5eb25e793c3c9da9a2; no voters: b6ce0bf86ac946d19dbd135886445059, f7a13548bd2d41d7a9a6b575a8d134e8
I20260807 19:42:23.337069 614 leader_election.cc:302] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67e9984475634f5eb25e793c3c9da9a2, f7a13548bd2d41d7a9a6b575a8d134e8; no voters:
I20260807 19:42:23.337344 752 raft_consensus.cc:2726] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:42:23.337399 753 raft_consensus.cc:2781] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:23.337826 753 raft_consensus.cc:686] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 1 LEADER]: Becoming Leader. State: Replica: f7a13548bd2d41d7a9a6b575a8d134e8, State: Running, Role: LEADER
I20260807 19:42:23.338307 549 raft_consensus.cc:2454] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7a13548bd2d41d7a9a6b575a8d134e8 in term 1.
I20260807 19:42:23.338375 753 consensus_queue.cc:237] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e9984475634f5eb25e793c3c9da9a2" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39679 } } peers { permanent_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40613 } } peers { permanent_uuid: "b6ce0bf86ac946d19dbd135886445059" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44743 } }
I20260807 19:42:23.342650 32735 catalog_manager.cc:5495] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7a13548bd2d41d7a9a6b575a8d134e8 (127.14.16.67). New cstate: current_term: 1 leader_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e9984475634f5eb25e793c3c9da9a2" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7a13548bd2d41d7a9a6b575a8d134e8" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 40613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6ce0bf86ac946d19dbd135886445059" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44743 } health_report { overall_health: UNKNOWN } } }
I20260807 19:42:23.380544 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.375109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.14.16.67:40613)
0807 19:42:23.375357 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.375362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.375380 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:42:23.375532 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.375535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.375664 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.375792 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:23.376133 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.376918 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.378282 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.378718 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.378820 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.379352 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:23.379894 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:23.380013 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.380125 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.380193 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:42:23.389014 549 raft_consensus.cc:1261] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059 [term 1 FOLLOWER]: Refusing update from remote peer f7a13548bd2d41d7a9a6b575a8d134e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:23.389043 413 raft_consensus.cc:1261] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [term 1 FOLLOWER]: Refusing update from remote peer f7a13548bd2d41d7a9a6b575a8d134e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:23.389943 758 consensus_queue.cc:1035] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ce0bf86ac946d19dbd135886445059" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:23.390308 753 consensus_queue.cc:1035] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67e9984475634f5eb25e793c3c9da9a2" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:23.418123 32737 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:23.418412 32737 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:23.422137 32737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6153ea49b9b4ba7a3c38cb153daa20c: Sending DeleteTablet for 3 replicas of tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3
I20260807 19:42:23.423333 393 tablet_service.cc:1514] Processing DeleteTablet for tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:23 UTC) from {username='slave'} at 127.0.0.1:46714
I20260807 19:42:23.423489 529 tablet_service.cc:1514] Processing DeleteTablet for tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:23 UTC) from {username='slave'} at 127.0.0.1:57746
I20260807 19:42:23.424464 661 tablet_service.cc:1514] Processing DeleteTablet for tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:23 UTC) from {username='slave'} at 127.0.0.1:36846
I20260807 19:42:23.430671 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 329
I20260807 19:42:23.431264 771 tablet_replica.cc:330] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2: stopping tablet replica
I20260807 19:42:23.431802 771 raft_consensus.cc:2229] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:23.432314 771 raft_consensus.cc:2258] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:23.435032 771 ts_tablet_manager.cc:1905] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:23.440667 773 tablet_replica.cc:330] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8: stopping tablet replica
I20260807 19:42:23.441212 773 raft_consensus.cc:2229] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:23.441956 772 tablet_replica.cc:330] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059: stopping tablet replica
I20260807 19:42:23.442448 741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.431920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.14.16.65:39679)
0807 19:42:23.432104 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.432110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.433063 (+ 953us) server_negotiation.cc:407] Connection header received
0807 19:42:23.433276 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.433280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.433388 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.433508 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.434184 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.435342 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.438423 (+ 3081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.439093 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.439214 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.440744 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.440756 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.440776 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.441598 (+ 822us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.441663 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.441691 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.441817 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.441935 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.442058 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:42:23.443189 772 raft_consensus.cc:2229] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:23.443411 455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:23.443485 773 raft_consensus.cc:2258] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:23.444321 772 raft_consensus.cc:2258] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:23.446674 773 ts_tablet_manager.cc:1905] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:23.448203 772 ts_tablet_manager.cc:1905] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:23.455335 772 ts_tablet_manager.cc:1918] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:23.455566 772 log.cc:1196] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/dc0b3a5f341b4d7da5d3cbe1dc4b79a3
I20260807 19:42:23.456221 772 ts_tablet_manager.cc:1939] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P b6ce0bf86ac946d19dbd135886445059: Deleting consensus metadata
I20260807 19:42:23.456868 773 ts_tablet_manager.cc:1918] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:23.457161 773 log.cc:1196] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/dc0b3a5f341b4d7da5d3cbe1dc4b79a3
I20260807 19:42:23.457938 773 ts_tablet_manager.cc:1939] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P f7a13548bd2d41d7a9a6b575a8d134e8: Deleting consensus metadata
I20260807 19:42:23.458418 32723 catalog_manager.cc:4841] TS b6ce0bf86ac946d19dbd135886445059 (127.14.16.66:44743): tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3 (table test-table [id=200a6d76979b4c2a915093990c92ce6b]) successfully deleted
I20260807 19:42:23.460134 32723 catalog_manager.cc:4841] TS f7a13548bd2d41d7a9a6b575a8d134e8 (127.14.16.67:40613): tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3 (table test-table [id=200a6d76979b4c2a915093990c92ce6b]) successfully deleted
I20260807 19:42:23.587132 771 ts_tablet_manager.cc:1918] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:23.587487 771 log.cc:1196] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/dc0b3a5f341b4d7da5d3cbe1dc4b79a3
I20260807 19:42:23.588130 771 ts_tablet_manager.cc:1939] T dc0b3a5f341b4d7da5d3cbe1dc4b79a3 P 67e9984475634f5eb25e793c3c9da9a2: Deleting consensus metadata
I20260807 19:42:23.588426 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 329
W20260807 19:42:23.609149 32722 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.16.65:39679 (error 108)
I20260807 19:42:23.609963 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 464
I20260807 19:42:23.611724 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.611139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39679 (local address 127.0.0.1:46736)
0807 19:42:23.611400 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.611527 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:39679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:42:23.611984 32722 catalog_manager.cc:4584] TS 67e9984475634f5eb25e793c3c9da9a2 (127.14.16.65:39679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc0b3a5f341b4d7da5d3cbe1dc4b79a3: Network error: Client connection negotiation failed: client connection to 127.14.16.65:39679: connect: Connection refused (error 111)
I20260807 19:42:23.617308 744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.610808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57758 (local address 127.14.16.66:44743)
0807 19:42:23.611010 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.611016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.611039 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:42:23.611242 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.611248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.611411 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.611529 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.612139 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.613064 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.614346 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.614751 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.614886 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.615989 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.615998 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.616012 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.616567 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.616613 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.616643 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.616777 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.616909 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.617008 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:42:23.618027 589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:23.665832 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.665350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39679 (local address 127.0.0.1:46738)
0807 19:42:23.665537 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.665635 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:39679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 19:42:23.667373 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.666792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39679 (local address 127.0.0.1:46752)
0807 19:42:23.667084 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.667167 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:39679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 19:42:23.709314 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 464
I20260807 19:42:23.728058 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.727462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39679 (local address 127.0.0.1:46766)
0807 19:42:23.727711 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.727836 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:39679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 19:42:23.729432 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.728952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39679 (local address 127.0.0.1:46772)
0807 19:42:23.729160 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.729258 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:39679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:42:23.729583 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 598
I20260807 19:42:23.737545 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.730354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36862 (local address 127.14.16.67:40613)
0807 19:42:23.730550 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.730563 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.730587 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:42:23.730670 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.730675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.730798 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.730931 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.731438 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.732665 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.733940 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:23.734669 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:23.734805 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:23.735912 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.735926 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.735949 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.736748 (+ 799us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.736827 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.736857 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.736969 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.737092 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.737206 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:42:23.738382 722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:23.810765 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.810270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39679 (local address 127.0.0.1:46780)
0807 19:42:23.810485 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.810613 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:39679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 19:42:23.812024 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.811627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39679 (local address 127.0.0.1:46794)
0807 19:42:23.811833 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.811907 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:39679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 19:42:23.861683 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 598
I20260807 19:42:23.884311 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 32706
I20260807 19:42:23.885430 300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:23.996099 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.994850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39679 (local address 127.0.0.1:46796)
0807 19:42:23.995211 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.995368 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.65:39679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 19:42:23.996207 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 32706
2026-08-07T19:42:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2883 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:24 dist-test-slave-07c2 krb5kdc[782](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:24 dist-test-slave-07c2 krb5kdc[782](info): set up 2 sockets
Aug 07 19:42:24 dist-test-slave-07c2 krb5kdc[782](info): commencing operation
krb5kdc: starting...
W20260807 19:42:25.151278 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.010s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:25 dist-test-slave-07c2 krb5kdc[782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:42:25.463725 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:41723
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:41303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:41723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:25.809891 795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:25.810169 795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:25.810261 795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:25.818132 795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:25.818244 795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:25.818296 795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:25.818353 795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:25.818397 795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:25.828464 795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:41303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:41723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:41723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:25.829752 795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:25.831007 795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:25.842800 802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:25.842825 801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:25.843492 804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:25.843778 795 server_base.cc:1034] running on GCE node
I20260807 19:42:25.844550 795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:25.845882 795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:25.847111 795 hybrid_clock.cc:648] HybridClock initialized: now 1786131745847076 us; error 49 us; skew 500 ppm
I20260807 19:42:25.847508 795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:25 dist-test-slave-07c2 krb5kdc[782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131745, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:25.850499 795 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:42:25.852084 795 webserver.cc:460] Webserver started at http://127.14.16.126:45405/ using document root <none> and password file <none>
I20260807 19:42:25.852753 795 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:25.852903 795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:25.853209 795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:25.856242 795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "045c34a471104ebe8ce82693328002b1"
format_stamp: "Formatted at 2026-08-07 19:42:25 on dist-test-slave-07c2"
server_key: "ae3f9bae1e8f96ba01859526aca400f7"
server_key_iv: "e33b5076ce3000150ac6b4f54d901799"
server_key_version: "encryptionkey@0"
I20260807 19:42:25.857008 795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "045c34a471104ebe8ce82693328002b1"
format_stamp: "Formatted at 2026-08-07 19:42:25 on dist-test-slave-07c2"
server_key: "ae3f9bae1e8f96ba01859526aca400f7"
server_key_iv: "e33b5076ce3000150ac6b4f54d901799"
server_key_version: "encryptionkey@0"
I20260807 19:42:25.861956 795 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:42:25.865360 811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:25.866528 795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 19:42:25.866720 795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "045c34a471104ebe8ce82693328002b1"
format_stamp: "Formatted at 2026-08-07 19:42:25 on dist-test-slave-07c2"
server_key: "ae3f9bae1e8f96ba01859526aca400f7"
server_key_iv: "e33b5076ce3000150ac6b4f54d901799"
server_key_version: "encryptionkey@0"
I20260807 19:42:25.866914 795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:25.880540 795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:25.884331 795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:25.884613 795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:25.903716 795 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:41723
I20260807 19:42:25.903736 863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:41723 every 8 connection(s)
I20260807 19:42:25.905131 795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:25.908851 864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:25.914891 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 795
I20260807 19:42:25.915088 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:25.915675 14401 external_mini_cluster.cc:1428] Setting key 8415b18434a5bc902bafbf0c868e2add
I20260807 19:42:25.919930 864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1: Bootstrap starting.
I20260807 19:42:25.923568 864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:25.924724 864 log.cc:824] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:25.928025 864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1: No bootstrap required, opened a new log
Aug 07 19:42:25 dist-test-slave-07c2 krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:42:25.936659 864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045c34a471104ebe8ce82693328002b1" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41723 } }
I20260807 19:42:25.937040 864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:25.937194 864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 045c34a471104ebe8ce82693328002b1, State: Initialized, Role: FOLLOWER
I20260807 19:42:25.937870 864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045c34a471104ebe8ce82693328002b1" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41723 } }
I20260807 19:42:25.938212 864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:25.938367 864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:25.938555 864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:25.939424 867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:25.917632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33478 (local address 127.14.16.126:41723)
0807 19:42:25.918412 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:25.918424 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:25.918463 (+ 39us) server_negotiation.cc:407] Connection header received
0807 19:42:25.920233 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:25.920257 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:25.920602 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:25.920911 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:25.922701 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:25.924139 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:25.925916 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:25.926627 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:25.926763 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:25.931764 (+ 5001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:25.931772 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:25.931789 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:25.931827 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:25.935023 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:25.935850 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:25.935858 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:25.935864 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:25.936030 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:25.936424 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:25.936430 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:25.936435 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:25.936979 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:25.937317 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:25.937725 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 19:42:25.938181 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":381,"threads_started":1}
I20260807 19:42:25.941192 864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045c34a471104ebe8ce82693328002b1" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41723 } }
I20260807 19:42:25.941680 864 leader_election.cc:302] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 045c34a471104ebe8ce82693328002b1; no voters:
I20260807 19:42:25.942407 864 leader_election.cc:288] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:25.942492 869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:25.943241 869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [term 1 LEADER]: Becoming Leader. State: Replica: 045c34a471104ebe8ce82693328002b1, State: Running, Role: LEADER
I20260807 19:42:25.943809 869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045c34a471104ebe8ce82693328002b1" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41723 } }
I20260807 19:42:25.944511 864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:25.946285 871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 045c34a471104ebe8ce82693328002b1. Latest consensus state: current_term: 1 leader_uuid: "045c34a471104ebe8ce82693328002b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045c34a471104ebe8ce82693328002b1" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41723 } } }
I20260807 19:42:25.946888 870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "045c34a471104ebe8ce82693328002b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045c34a471104ebe8ce82693328002b1" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 41723 } } }
I20260807 19:42:25.947204 870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:25.947204 871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:25.950274 874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:25.957557 874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:25.966310 874 catalog_manager.cc:1347] Generated new cluster ID: 7d86271cec364676ab6549a323653cb7
I20260807 19:42:25.966428 874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:25.984706 874 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:25.986575 874 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:25.998771 874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1: Generated new TSK 0
I20260807 19:42:25.999924 874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:42:26.084492 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41723
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:41303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:26.427850 890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:26.428114 890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:26.428292 890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:26.435346 890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:26.435487 890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:26.435685 890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:42:26.446770 890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:41303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:26.448240 890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:26.449703 890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:26.464656 898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:26.464771 896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:26.465202 895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:26.466004 890 server_base.cc:1034] running on GCE node
I20260807 19:42:26.466666 890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:26.467741 890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:26.468981 890 hybrid_clock.cc:648] HybridClock initialized: now 1786131746468926 us; error 43 us; skew 500 ppm
I20260807 19:42:26.469317 890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:26 dist-test-slave-07c2 krb5kdc[782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131746, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:26.472765 890 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:42:26.474336 890 webserver.cc:460] Webserver started at http://127.14.16.65:46787/ using document root <none> and password file <none>
I20260807 19:42:26.475137 890 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:26.475337 890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:26.475692 890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:26.478737 890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "956c3a9b4c574fa6ac5f521f280af773"
format_stamp: "Formatted at 2026-08-07 19:42:26 on dist-test-slave-07c2"
server_key: "55c7479243343fd395be978847f38ea2"
server_key_iv: "f9725afc4dd0b5eca6d0dff648c1654f"
server_key_version: "encryptionkey@0"
I20260807 19:42:26.479604 890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "956c3a9b4c574fa6ac5f521f280af773"
format_stamp: "Formatted at 2026-08-07 19:42:26 on dist-test-slave-07c2"
server_key: "55c7479243343fd395be978847f38ea2"
server_key_iv: "f9725afc4dd0b5eca6d0dff648c1654f"
server_key_version: "encryptionkey@0"
I20260807 19:42:26.484902 890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:26.488482 905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:26.489959 890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:42:26.490182 890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "956c3a9b4c574fa6ac5f521f280af773"
format_stamp: "Formatted at 2026-08-07 19:42:26 on dist-test-slave-07c2"
server_key: "55c7479243343fd395be978847f38ea2"
server_key_iv: "f9725afc4dd0b5eca6d0dff648c1654f"
server_key_version: "encryptionkey@0"
I20260807 19:42:26.490420 890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:26.503661 890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:26.507733 890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:26.508034 890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:26.509022 890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:26.510347 890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:26.510453 890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:26.510607 890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:26.510680 890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:26.545323 890 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:34795
I20260807 19:42:26.545354 1019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:34795 every 8 connection(s)
I20260807 19:42:26.546768 890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:26.547857 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 890
I20260807 19:42:26.548055 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:26.548683 14401 external_mini_cluster.cc:1428] Setting key 7fed6db8691e15f9bf94bda26dd9a488
Aug 07 19:42:26 dist-test-slave-07c2 krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131746, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:42:26.568151 1022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:26.550035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:60071 (local address 127.14.16.126:41723)
0807 19:42:26.550527 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:26.550533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:26.552082 (+ 1549us) server_negotiation.cc:407] Connection header received
0807 19:42:26.553649 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:26.553655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:26.553739 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:26.553872 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:26.555445 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:26.556204 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:26.557778 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:26.558231 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:26.558368 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:26.561937 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:26.561942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:26.561945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:26.561979 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:26.564625 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:26.565412 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:26.565416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:26.565419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:26.565542 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:26.566004 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:26.566013 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:26.566015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:26.566282 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:26.566444 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:26.567502 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0807 19:42:26.567758 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":235,"threads_started":1}
I20260807 19:42:26.569169 1023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:26.550721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41723 (local address 127.14.16.65:60071)
0807 19:42:26.551532 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:26.551576 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:26.553390 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:26.554108 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:26.554116 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:26.554655 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:26.555219 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:26.555236 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:26.556362 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:26.556367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:26.557615 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:26.557632 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:26.558481 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:26.558490 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:26.558733 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:26.559489 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:26.559512 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:26.561703 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:26.564776 (+ 3073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:26.564787 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:26.564807 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:26.565273 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:26.565658 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:26.565663 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:26.565667 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:26.565874 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:26.566416 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:26.566424 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:26.567025 (+ 601us) client_negotiation.cc:769] Sending connection context
0807 19:42:26.567347 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 19:42:26.567942 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":340,"threads_started":1}
I20260807 19:42:26.570869 1020 heartbeater.cc:347] Connected to a master server at 127.14.16.126:41723
I20260807 19:42:26.571208 1020 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:26.572178 1020 heartbeater.cc:511] Master 127.14.16.126:41723 requested a full tablet report, sending...
I20260807 19:42:26.574374 829 ts_manager.cc:194] Registered new tserver with Master: 956c3a9b4c574fa6ac5f521f280af773 (127.14.16.65:34795)
I20260807 19:42:26.576025 829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:60071
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:42:26.619228 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41723
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:41303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:26.958225 1026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:26.958523 1026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:26.958699 1026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:26.966051 1026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:26.966174 1026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:26.966388 1026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:42:26.976624 1026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:41303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:26.978075 1026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:26.979529 1026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:26.993170 1032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:26.993257 1031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:26.993800 1034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:26.994009 1026 server_base.cc:1034] running on GCE node
I20260807 19:42:26.994661 1026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:26.995688 1026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:26.996884 1026 hybrid_clock.cc:648] HybridClock initialized: now 1786131746996826 us; error 51 us; skew 500 ppm
I20260807 19:42:26.997153 1026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:27 dist-test-slave-07c2 krb5kdc[782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131746, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:27.004120 1026 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:42:27.005645 1026 webserver.cc:460] Webserver started at http://127.14.16.66:35909/ using document root <none> and password file <none>
I20260807 19:42:27.006372 1026 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:27.006510 1026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:27.006865 1026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:27.010036 1026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "6508f10e91f546258441c0f37a2a0c8d"
format_stamp: "Formatted at 2026-08-07 19:42:27 on dist-test-slave-07c2"
server_key: "6a2f3f80ade3315ed8ffb647a6e54d74"
server_key_iv: "d8277ff4108dee55dfea1782e6753ac9"
server_key_version: "encryptionkey@0"
I20260807 19:42:27.010840 1026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "6508f10e91f546258441c0f37a2a0c8d"
format_stamp: "Formatted at 2026-08-07 19:42:27 on dist-test-slave-07c2"
server_key: "6a2f3f80ade3315ed8ffb647a6e54d74"
server_key_iv: "d8277ff4108dee55dfea1782e6753ac9"
server_key_version: "encryptionkey@0"
I20260807 19:42:27.016369 1026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 19:42:27.019878 1041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:27.021170 1026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:42:27.021360 1026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "6508f10e91f546258441c0f37a2a0c8d"
format_stamp: "Formatted at 2026-08-07 19:42:27 on dist-test-slave-07c2"
server_key: "6a2f3f80ade3315ed8ffb647a6e54d74"
server_key_iv: "d8277ff4108dee55dfea1782e6753ac9"
server_key_version: "encryptionkey@0"
I20260807 19:42:27.021565 1026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:27.048261 1026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:27.052947 1026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:27.053252 1026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:27.054304 1026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:27.055732 1026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:27.055843 1026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:27.055989 1026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:27.056061 1026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:27.091085 1026 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:41351
I20260807 19:42:27.091223 1157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:41351 every 8 connection(s)
I20260807 19:42:27.092455 1026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:27.101768 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 1026
I20260807 19:42:27.101987 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:27.102561 14401 external_mini_cluster.cc:1428] Setting key 400515aa87c91b74f2d59c6d8ccf675e
Aug 07 19:42:27 dist-test-slave-07c2 krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131746, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:42:27.115064 1160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.095207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:41321 (local address 127.14.16.126:41723)
0807 19:42:27.095691 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:27.095696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:27.096596 (+ 900us) server_negotiation.cc:407] Connection header received
0807 19:42:27.098485 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:27.098491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:27.098589 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:27.098721 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:27.100354 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.101131 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.103555 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.104205 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.104337 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.108863 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:27.108869 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:27.108872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:27.108907 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:27.111704 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:27.112483 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:27.112488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:27.112490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:27.112625 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:27.113099 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:27.113103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:27.113105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:27.113404 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:27.113652 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:27.114420 (+ 768us) server_negotiation.cc:299] Negotiation successful
0807 19:42:27.114661 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":374,"thread_start_us":268,"threads_started":1}
I20260807 19:42:27.116086 1161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.095711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41723 (local address 127.14.16.66:41321)
0807 19:42:27.096433 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:27.096470 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:27.098169 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:27.098961 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:27.098969 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:27.099606 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:27.100173 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.100190 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.101307 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.101331 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.103382 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.103401 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.104435 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.104440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.104776 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.105616 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:27.105653 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:27.108532 (+ 2879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:27.111851 (+ 3319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:27.111861 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:27.111880 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:27.112334 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:27.112740 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:27.112745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:27.112749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:27.112955 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:27.113620 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:27.113629 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:27.114012 (+ 383us) client_negotiation.cc:769] Sending connection context
0807 19:42:27.114311 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 19:42:27.114897 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":274,"threads_started":1}
I20260807 19:42:27.117690 1158 heartbeater.cc:347] Connected to a master server at 127.14.16.126:41723
I20260807 19:42:27.118016 1158 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:27.118728 1158 heartbeater.cc:511] Master 127.14.16.126:41723 requested a full tablet report, sending...
I20260807 19:42:27.120260 829 ts_manager.cc:194] Registered new tserver with Master: 6508f10e91f546258441c0f37a2a0c8d (127.14.16.66:41351)
I20260807 19:42:27.121100 829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:41321
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:42:27.171123 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41723
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:41303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:27.523641 1164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:27.523952 1164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:27.524129 1164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:27.531260 1164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:27.531432 1164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:27.531635 1164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:42:27.541237 1164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:41303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:41723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:27.542595 1164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:27.543879 1164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:27.557581 1172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:27.557554 1170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:27.557520 1169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:27.558226 1164 server_base.cc:1034] running on GCE node
I20260807 19:42:27.558845 1164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:27.559836 1164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:27.561066 1164 hybrid_clock.cc:648] HybridClock initialized: now 1786131747561012 us; error 49 us; skew 500 ppm
I20260807 19:42:27.561380 1164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:27 dist-test-slave-07c2 krb5kdc[782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131747, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:27.564641 1164 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:42:27.566007 1164 webserver.cc:460] Webserver started at http://127.14.16.67:46803/ using document root <none> and password file <none>
I20260807 19:42:27.566640 1164 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:27.566744 1164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:27.567004 1164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:27.569954 1164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "95ee2beebce244d498ca36f72b25d7dc"
format_stamp: "Formatted at 2026-08-07 19:42:27 on dist-test-slave-07c2"
server_key: "676f70531fabea43ca39131a90b79699"
server_key_iv: "5c655c2da00d135685b55dca792a6ab6"
server_key_version: "encryptionkey@0"
I20260807 19:42:27.570761 1164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "95ee2beebce244d498ca36f72b25d7dc"
format_stamp: "Formatted at 2026-08-07 19:42:27 on dist-test-slave-07c2"
server_key: "676f70531fabea43ca39131a90b79699"
server_key_iv: "5c655c2da00d135685b55dca792a6ab6"
server_key_version: "encryptionkey@0"
I20260807 19:42:27.578258 1164 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260807 19:42:27.579198 1020 heartbeater.cc:503] Master 127.14.16.126:41723 was elected leader, sending a full tablet report...
I20260807 19:42:27.582787 1179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:27.584137 1164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:27.584317 1164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "95ee2beebce244d498ca36f72b25d7dc"
format_stamp: "Formatted at 2026-08-07 19:42:27 on dist-test-slave-07c2"
server_key: "676f70531fabea43ca39131a90b79699"
server_key_iv: "5c655c2da00d135685b55dca792a6ab6"
server_key_version: "encryptionkey@0"
I20260807 19:42:27.584532 1164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:27.617416 1164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:27.621956 1164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:27.622273 1164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:27.623417 1164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:27.624845 1164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:27.624981 1164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:27.625126 1164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:27.625211 1164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:27.660038 1164 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:41939
I20260807 19:42:27.660058 1295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:41939 every 8 connection(s)
I20260807 19:42:27.661504 1164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:27.663815 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 1164
I20260807 19:42:27.664019 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:27.664602 14401 external_mini_cluster.cc:1428] Setting key 4d455a793581c069e0133930ba9dbcb3
Aug 07 19:42:27 dist-test-slave-07c2 krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131747, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:42:27.681859 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.664154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:42177 (local address 127.14.16.126:41723)
0807 19:42:27.664649 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:27.664655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:27.665793 (+ 1138us) server_negotiation.cc:407] Connection header received
0807 19:42:27.667922 (+ 2129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:27.667927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:27.668020 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:27.668135 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:27.669608 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.670462 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.672032 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.672428 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.672551 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.675840 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:27.675844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:27.675847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:27.675880 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:27.678564 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:27.679576 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:27.679584 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:27.679588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:27.679829 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:27.680292 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:27.680297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:27.680299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:27.680636 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:27.680767 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:27.681402 (+ 635us) server_negotiation.cc:299] Negotiation successful
0807 19:42:27.681575 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":214,"threads_started":1}
I20260807 19:42:27.682637 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.664813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:41723 (local address 127.14.16.67:42177)
0807 19:42:27.665636 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:27.665674 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:27.667671 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:27.668368 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:27.668376 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:27.668904 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:27.669444 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.669460 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.670615 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.670619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.671894 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.671906 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.672658 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.672666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.672886 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.673490 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:27.673511 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:27.675661 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:27.678719 (+ 3058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:27.678727 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:27.678741 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:27.679437 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:27.679955 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:27.679961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:27.679965 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:27.680151 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:27.680767 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:27.680772 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:27.681050 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 19:42:27.681262 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 19:42:27.681606 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":365,"threads_started":1}
I20260807 19:42:27.683985 1296 heartbeater.cc:347] Connected to a master server at 127.14.16.126:41723
I20260807 19:42:27.684254 1296 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:27.685050 1296 heartbeater.cc:511] Master 127.14.16.126:41723 requested a full tablet report, sending...
I20260807 19:42:27.686616 829 ts_manager.cc:194] Registered new tserver with Master: 95ee2beebce244d498ca36f72b25d7dc (127.14.16.67:41939)
I20260807 19:42:27.687495 829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:42177
I20260807 19:42:27.692158 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:42:27.709369 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.696499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33484 (local address 127.14.16.126:41723)
0807 19:42:27.696741 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:27.696748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:27.697062 (+ 314us) server_negotiation.cc:407] Connection header received
0807 19:42:27.697182 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:27.697186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:27.697261 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:27.697342 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:27.698734 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.699537 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.700793 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.701214 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.701312 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.704002 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:27.704007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:27.704010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:27.704043 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:27.706530 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:27.707392 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:27.707396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:27.707398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:27.707519 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:27.707942 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:27.707945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:27.707947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:27.708252 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:27.708454 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:27.708833 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 19:42:27.709036 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 19:42:27.713279 829 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:27.720384 829 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:27.750118 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.733108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.14.16.65:34795)
0807 19:42:27.740930 (+ 7822us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:27.740941 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:27.740975 (+ 34us) server_negotiation.cc:407] Connection header received
0807 19:42:27.741081 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:27.741089 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:27.741357 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:27.741550 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:27.742662 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.744350 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.747060 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.748620 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.748766 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.748994 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:27.749577 (+ 583us) server_negotiation.cc:299] Negotiation successful
0807 19:42:27.749776 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7616,"thread_start_us":7448,"threads_started":1}
I20260807 19:42:27.750196 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.733008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:34795 (local address 127.0.0.1:45466)
0807 19:42:27.740106 (+ 7098us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:27.740176 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:27.740405 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:27.741892 (+ 1487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:27.741902 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:27.741939 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:27.742454 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.742471 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.744522 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.744528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.746872 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.746889 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.748969 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.748977 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.749305 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.749378 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 19:42:27.749519 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 19:42:27.749721 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6741,"mutex_wait_us":70,"thread_start_us":8427,"threads_started":1}
I20260807 19:42:27.752432 1309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.733162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:41351 (local address 127.0.0.1:59788)
0807 19:42:27.741155 (+ 7993us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:27.741227 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:27.741345 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:27.743520 (+ 2175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:27.743525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:27.743550 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:27.743936 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.743947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.746965 (+ 3018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.746969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.749264 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.749278 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.751156 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.751162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.751368 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.751400 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:42:27.751488 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 19:42:27.752054 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7878,"mutex_wait_us":1,"thread_start_us":6396,"threads_started":1}
I20260807 19:42:27.753284 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.733707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.14.16.66:41351)
0807 19:42:27.742662 (+ 8955us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:27.742670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:27.742695 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:42:27.742804 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:27.742813 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:27.743030 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:27.743224 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:27.744150 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.746800 (+ 2650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.749427 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.750906 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.751060 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.751271 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:27.751684 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 19:42:27.751863 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8797,"thread_start_us":8674,"threads_started":1}
I20260807 19:42:27.755774 954 tablet_service.cc:1467] Processing CreateTablet for tablet 017a0915d3594084b210a2f29f59afd4 (DEFAULT_TABLE table=test-table [id=f5b62b4fd2ff46cf8e4ba68bf67078c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:27.756598 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.742796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:41939 (local address 127.0.0.1:41076)
0807 19:42:27.748949 (+ 6153us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:27.748966 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:27.749100 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:27.750039 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:27.750044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:27.750069 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:27.750512 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.750527 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.752611 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.752618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.754488 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.754501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.755806 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.755814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.755995 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.756027 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:42:27.756137 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 19:42:27.756266 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6015,"thread_start_us":5851,"threads_started":1}
I20260807 19:42:27.757009 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.742190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41076 (local address 127.14.16.67:41939)
0807 19:42:27.748611 (+ 6421us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:27.748633 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:27.749079 (+ 446us) server_negotiation.cc:407] Connection header received
0807 19:42:27.749306 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:27.749317 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:27.749580 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:27.749769 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:27.750716 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.752351 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.754640 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.755605 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.755742 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.755959 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:27.756198 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 19:42:27.756359 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6218,"thread_start_us":6069,"threads_started":1}
I20260807 19:42:27.756708 1090 tablet_service.cc:1467] Processing CreateTablet for tablet 017a0915d3594084b210a2f29f59afd4 (DEFAULT_TABLE table=test-table [id=f5b62b4fd2ff46cf8e4ba68bf67078c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:27.757370 954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 017a0915d3594084b210a2f29f59afd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:27.758018 1090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 017a0915d3594084b210a2f29f59afd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:27.760295 1229 tablet_service.cc:1467] Processing CreateTablet for tablet 017a0915d3594084b210a2f29f59afd4 (DEFAULT_TABLE table=test-table [id=f5b62b4fd2ff46cf8e4ba68bf67078c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:27.761335 1229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 017a0915d3594084b210a2f29f59afd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:27.772114 1315 tablet_bootstrap.cc:492] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc: Bootstrap starting.
I20260807 19:42:27.773468 1316 tablet_bootstrap.cc:492] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d: Bootstrap starting.
I20260807 19:42:27.774482 1317 tablet_bootstrap.cc:492] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773: Bootstrap starting.
I20260807 19:42:27.777037 1316 tablet_bootstrap.cc:654] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:27.778092 1317 tablet_bootstrap.cc:654] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:27.778093 1315 tablet_bootstrap.cc:654] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:27.778201 1316 log.cc:824] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:27.779345 1315 log.cc:824] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:27.779342 1317 log.cc:824] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:27.781613 1316 tablet_bootstrap.cc:492] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d: No bootstrap required, opened a new log
I20260807 19:42:27.781914 1316 ts_tablet_manager.cc:1397] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260807 19:42:27.782043 1315 tablet_bootstrap.cc:492] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc: No bootstrap required, opened a new log
I20260807 19:42:27.782303 1315 ts_tablet_manager.cc:1397] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 19:42:27.782646 1317 tablet_bootstrap.cc:492] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773: No bootstrap required, opened a new log
I20260807 19:42:27.782907 1317 ts_tablet_manager.cc:1397] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260807 19:42:27.790261 1315 raft_consensus.cc:348] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956c3a9b4c574fa6ac5f521f280af773" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34795 } } peers { permanent_uuid: "6508f10e91f546258441c0f37a2a0c8d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 41351 } } peers { permanent_uuid: "95ee2beebce244d498ca36f72b25d7dc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41939 } }
I20260807 19:42:27.790648 1315 raft_consensus.cc:374] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:27.790753 1315 raft_consensus.cc:729] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95ee2beebce244d498ca36f72b25d7dc, State: Initialized, Role: FOLLOWER
I20260807 19:42:27.791440 1315 consensus_queue.cc:260] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956c3a9b4c574fa6ac5f521f280af773" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34795 } } peers { permanent_uuid: "6508f10e91f546258441c0f37a2a0c8d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 41351 } } peers { permanent_uuid: "95ee2beebce244d498ca36f72b25d7dc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41939 } }
I20260807 19:42:27.792541 1296 heartbeater.cc:503] Master 127.14.16.126:41723 was elected leader, sending a full tablet report...
I20260807 19:42:27.793278 1315 ts_tablet_manager.cc:1428] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260807 19:42:27.793051 1316 raft_consensus.cc:348] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956c3a9b4c574fa6ac5f521f280af773" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34795 } } peers { permanent_uuid: "6508f10e91f546258441c0f37a2a0c8d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 41351 } } peers { permanent_uuid: "95ee2beebce244d498ca36f72b25d7dc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41939 } }
I20260807 19:42:27.793514 1316 raft_consensus.cc:374] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:27.793672 1316 raft_consensus.cc:729] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6508f10e91f546258441c0f37a2a0c8d, State: Initialized, Role: FOLLOWER
I20260807 19:42:27.793823 1317 raft_consensus.cc:348] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956c3a9b4c574fa6ac5f521f280af773" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34795 } } peers { permanent_uuid: "6508f10e91f546258441c0f37a2a0c8d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 41351 } } peers { permanent_uuid: "95ee2beebce244d498ca36f72b25d7dc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41939 } }
I20260807 19:42:27.794253 1317 raft_consensus.cc:374] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:27.794428 1317 raft_consensus.cc:729] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 956c3a9b4c574fa6ac5f521f280af773, State: Initialized, Role: FOLLOWER
I20260807 19:42:27.794294 1316 consensus_queue.cc:260] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956c3a9b4c574fa6ac5f521f280af773" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34795 } } peers { permanent_uuid: "6508f10e91f546258441c0f37a2a0c8d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 41351 } } peers { permanent_uuid: "95ee2beebce244d498ca36f72b25d7dc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41939 } }
I20260807 19:42:27.795069 1317 consensus_queue.cc:260] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956c3a9b4c574fa6ac5f521f280af773" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34795 } } peers { permanent_uuid: "6508f10e91f546258441c0f37a2a0c8d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 41351 } } peers { permanent_uuid: "95ee2beebce244d498ca36f72b25d7dc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41939 } }
I20260807 19:42:27.796087 1158 heartbeater.cc:503] Master 127.14.16.126:41723 was elected leader, sending a full tablet report...
I20260807 19:42:27.796866 1316 ts_tablet_manager.cc:1428] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260807 19:42:27.799789 1317 ts_tablet_manager.cc:1428] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773: Time spent starting tablet: real 0.017s user 0.014s sys 0.001s
W20260807 19:42:27.844511 1159 tablet.cc:2369] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:27.854733 1321 raft_consensus.cc:482] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:27.855070 1321 raft_consensus.cc:504] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956c3a9b4c574fa6ac5f521f280af773" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34795 } } peers { permanent_uuid: "6508f10e91f546258441c0f37a2a0c8d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 41351 } } peers { permanent_uuid: "95ee2beebce244d498ca36f72b25d7dc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41939 } }
I20260807 19:42:27.856937 1321 leader_election.cc:288] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 956c3a9b4c574fa6ac5f521f280af773 (127.14.16.65:34795), 6508f10e91f546258441c0f37a2a0c8d (127.14.16.66:41351)
I20260807 19:42:27.863574 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.857327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:56011 (local address 127.14.16.65:34795)
0807 19:42:27.857551 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:27.857558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:27.857994 (+ 436us) server_negotiation.cc:407] Connection header received
0807 19:42:27.858208 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:27.858212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:27.858282 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:27.858401 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:27.858882 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.859908 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.861851 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.862624 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.862803 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.862876 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:27.863170 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 19:42:27.863321 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:42:27.863703 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.857383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:34795 (local address 127.14.16.67:56011)
0807 19:42:27.857834 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:27.857864 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:27.858025 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:27.858448 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:27.858452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:27.858532 (+ 80us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:27.858743 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.858752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.860037 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.860045 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.861688 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.861701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.862791 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.862797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.862995 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.863022 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 19:42:27.863143 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 19:42:27.863302 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":53,"spinlock_wait_cycles":22272}
I20260807 19:42:27.863783 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.857530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:49691 (local address 127.14.16.66:41351)
0807 19:42:27.857788 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:27.857797 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:27.858381 (+ 584us) server_negotiation.cc:407] Connection header received
0807 19:42:27.858696 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:27.858703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:27.858790 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:27.858879 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:27.859571 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.860317 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.862056 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.862830 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.863004 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.863074 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:27.863386 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 19:42:27.863514 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:42:27.863909 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.857449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:41351 (local address 127.14.16.67:49691)
0807 19:42:27.858254 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:27.858272 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:27.858485 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:27.858915 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:27.858919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:27.858949 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:27.859345 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.859357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.860472 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.860477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.861901 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.861913 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.863033 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.863039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.863192 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.863215 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:42:27.863359 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 19:42:27.863484 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":2,"thread_start_us":417,"threads_started":1}
I20260807 19:42:27.864835 1111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "017a0915d3594084b210a2f29f59afd4" candidate_uuid: "95ee2beebce244d498ca36f72b25d7dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6508f10e91f546258441c0f37a2a0c8d" is_pre_election: true
I20260807 19:42:27.864948 975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "017a0915d3594084b210a2f29f59afd4" candidate_uuid: "95ee2beebce244d498ca36f72b25d7dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956c3a9b4c574fa6ac5f521f280af773" is_pre_election: true
I20260807 19:42:27.865372 1111 raft_consensus.cc:2454] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95ee2beebce244d498ca36f72b25d7dc in term 0.
I20260807 19:42:27.865460 975 raft_consensus.cc:2454] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95ee2beebce244d498ca36f72b25d7dc in term 0.
I20260807 19:42:27.866081 1180 leader_election.cc:302] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6508f10e91f546258441c0f37a2a0c8d, 95ee2beebce244d498ca36f72b25d7dc; no voters:
I20260807 19:42:27.866478 1321 raft_consensus.cc:2781] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:27.866632 1321 raft_consensus.cc:482] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:27.866745 1321 raft_consensus.cc:3037] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:27.868989 1321 raft_consensus.cc:504] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956c3a9b4c574fa6ac5f521f280af773" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34795 } } peers { permanent_uuid: "6508f10e91f546258441c0f37a2a0c8d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 41351 } } peers { permanent_uuid: "95ee2beebce244d498ca36f72b25d7dc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41939 } }
I20260807 19:42:27.869536 1321 leader_election.cc:288] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [CANDIDATE]: Term 1 election: Requested vote from peers 956c3a9b4c574fa6ac5f521f280af773 (127.14.16.65:34795), 6508f10e91f546258441c0f37a2a0c8d (127.14.16.66:41351)
I20260807 19:42:27.869941 975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "017a0915d3594084b210a2f29f59afd4" candidate_uuid: "95ee2beebce244d498ca36f72b25d7dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "956c3a9b4c574fa6ac5f521f280af773"
I20260807 19:42:27.870000 1111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "017a0915d3594084b210a2f29f59afd4" candidate_uuid: "95ee2beebce244d498ca36f72b25d7dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6508f10e91f546258441c0f37a2a0c8d"
I20260807 19:42:27.870152 975 raft_consensus.cc:3037] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:27.870221 1111 raft_consensus.cc:3037] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:27.872507 1111 raft_consensus.cc:2454] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95ee2beebce244d498ca36f72b25d7dc in term 1.
I20260807 19:42:27.872653 975 raft_consensus.cc:2454] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95ee2beebce244d498ca36f72b25d7dc in term 1.
I20260807 19:42:27.872931 1180 leader_election.cc:302] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6508f10e91f546258441c0f37a2a0c8d, 95ee2beebce244d498ca36f72b25d7dc; no voters:
I20260807 19:42:27.873208 1321 raft_consensus.cc:2781] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:27.873685 1321 raft_consensus.cc:686] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 1 LEADER]: Becoming Leader. State: Replica: 95ee2beebce244d498ca36f72b25d7dc, State: Running, Role: LEADER
I20260807 19:42:27.874257 1321 consensus_queue.cc:237] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956c3a9b4c574fa6ac5f521f280af773" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34795 } } peers { permanent_uuid: "6508f10e91f546258441c0f37a2a0c8d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 41351 } } peers { permanent_uuid: "95ee2beebce244d498ca36f72b25d7dc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41939 } }
I20260807 19:42:27.878734 829 catalog_manager.cc:5495] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 95ee2beebce244d498ca36f72b25d7dc (127.14.16.67). New cstate: current_term: 1 leader_uuid: "95ee2beebce244d498ca36f72b25d7dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "956c3a9b4c574fa6ac5f521f280af773" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6508f10e91f546258441c0f37a2a0c8d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 41351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95ee2beebce244d498ca36f72b25d7dc" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 41939 } health_report { overall_health: HEALTHY } } }
I20260807 19:42:27.901290 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.896172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41080 (local address 127.14.16.67:41939)
0807 19:42:27.896358 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:27.896363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:27.896503 (+ 140us) server_negotiation.cc:407] Connection header received
0807 19:42:27.896684 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:27.896687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:27.896780 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:27.896906 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:27.897252 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.897961 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.899269 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.899711 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.899816 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.900302 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:27.900795 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:27.900917 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:27.901055 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 19:42:27.901095 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:42:27.909897 975 raft_consensus.cc:1261] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773 [term 1 FOLLOWER]: Refusing update from remote peer 95ee2beebce244d498ca36f72b25d7dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:27.909988 1111 raft_consensus.cc:1261] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d [term 1 FOLLOWER]: Refusing update from remote peer 95ee2beebce244d498ca36f72b25d7dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:27.910910 1325 consensus_queue.cc:1035] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "956c3a9b4c574fa6ac5f521f280af773" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 34795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:27.911332 1321 consensus_queue.cc:1035] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6508f10e91f546258441c0f37a2a0c8d" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 41351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:42:27.913604 1297 tablet.cc:2369] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:27.940537 829 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:27.940832 829 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:27.944646 829 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 045c34a471104ebe8ce82693328002b1: Sending DeleteTablet for 3 replicas of tablet 017a0915d3594084b210a2f29f59afd4
I20260807 19:42:27.945907 954 tablet_service.cc:1514] Processing DeleteTablet for tablet 017a0915d3594084b210a2f29f59afd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:27 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:45466
I20260807 19:42:27.945927 1090 tablet_service.cc:1514] Processing DeleteTablet for tablet 017a0915d3594084b210a2f29f59afd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:27 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:59788
I20260807 19:42:27.948273 1229 tablet_service.cc:1514] Processing DeleteTablet for tablet 017a0915d3594084b210a2f29f59afd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:27 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:41076
I20260807 19:42:27.949644 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 890
I20260807 19:42:27.959865 1340 tablet_replica.cc:330] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc: stopping tablet replica
I20260807 19:42:27.960634 1340 raft_consensus.cc:2229] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:27.961647 1340 raft_consensus.cc:2258] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:27.962582 1338 tablet_replica.cc:330] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773: stopping tablet replica
I20260807 19:42:27.963114 1338 raft_consensus.cc:2229] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:27.963799 1338 raft_consensus.cc:2258] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:27.964314 1340 ts_tablet_manager.cc:1905] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:27.964612 1339 tablet_replica.cc:330] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d: stopping tablet replica
I20260807 19:42:27.965091 1339 raft_consensus.cc:2229] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:27.965637 1339 raft_consensus.cc:2258] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:27.967818 1339 ts_tablet_manager.cc:1905] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:27.967892 1338 ts_tablet_manager.cc:1905] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:27.972440 1340 ts_tablet_manager.cc:1918] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:27.972626 1340 log.cc:1196] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/017a0915d3594084b210a2f29f59afd4
I20260807 19:42:27.973187 1340 ts_tablet_manager.cc:1939] T 017a0915d3594084b210a2f29f59afd4 P 95ee2beebce244d498ca36f72b25d7dc: Deleting consensus metadata
I20260807 19:42:27.975096 815 catalog_manager.cc:4841] TS 95ee2beebce244d498ca36f72b25d7dc (127.14.16.67:41939): tablet 017a0915d3594084b210a2f29f59afd4 (table test-table [id=f5b62b4fd2ff46cf8e4ba68bf67078c1]) successfully deleted
I20260807 19:42:27.975433 1339 ts_tablet_manager.cc:1918] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:27.975687 1339 log.cc:1196] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/017a0915d3594084b210a2f29f59afd4
I20260807 19:42:27.976307 1339 ts_tablet_manager.cc:1939] T 017a0915d3594084b210a2f29f59afd4 P 6508f10e91f546258441c0f37a2a0c8d: Deleting consensus metadata
I20260807 19:42:27.977551 1338 ts_tablet_manager.cc:1918] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:27.977727 1338 log.cc:1196] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/017a0915d3594084b210a2f29f59afd4
I20260807 19:42:27.978492 1338 ts_tablet_manager.cc:1939] T 017a0915d3594084b210a2f29f59afd4 P 956c3a9b4c574fa6ac5f521f280af773: Deleting consensus metadata
I20260807 19:42:27.978433 812 catalog_manager.cc:4841] TS 6508f10e91f546258441c0f37a2a0c8d (127.14.16.66:41351): tablet 017a0915d3594084b210a2f29f59afd4 (table test-table [id=f5b62b4fd2ff46cf8e4ba68bf67078c1]) successfully deleted
Aug 07 19:42:27 dist-test-slave-07c2 krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:42:27.980281 815 catalog_manager.cc:4841] TS 956c3a9b4c574fa6ac5f521f280af773 (127.14.16.65:34795): tablet 017a0915d3594084b210a2f29f59afd4 (table test-table [id=f5b62b4fd2ff46cf8e4ba68bf67078c1]) successfully deleted
I20260807 19:42:27.987229 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.951032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45468 (local address 127.14.16.65:34795)
0807 19:42:27.951230 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:27.951236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:27.955039 (+ 3803us) server_negotiation.cc:407] Connection header received
0807 19:42:27.955222 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:27.955227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:27.955343 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:27.955457 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:27.962834 (+ 7377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.964180 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.970827 (+ 6647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.971556 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.971675 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.980306 (+ 8631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:27.980321 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:27.980339 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:27.980382 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:27.984193 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:27.984950 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:27.984961 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:27.984964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:27.985134 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:27.985519 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:27.985525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:27.985528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:27.986007 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:27.986318 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:27.986590 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 19:42:27.986830 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:42:27.988102 1015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:28.078706 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 890
I20260807 19:42:28.106472 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 1026
Aug 07 19:42:28 dist-test-slave-07c2 krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:42:28.124321 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:28.107334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.14.16.66:41351)
0807 19:42:28.107564 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:28.107571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:28.107857 (+ 286us) server_negotiation.cc:407] Connection header received
0807 19:42:28.108044 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:28.108052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:28.108138 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:28.108258 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:28.109523 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:28.110670 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:28.111983 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:28.112620 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:28.112760 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:28.117035 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:28.117053 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:28.117075 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:28.117132 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:28.121261 (+ 4129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:28.121995 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:28.122006 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:28.122010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:28.122188 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:28.122546 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:28.122551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:28.122554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:28.123020 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:28.123354 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:28.123655 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 19:42:28.123891 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:42:28.125113 1153 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:28.222136 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 1026
I20260807 19:42:28.248123 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 1164
Aug 07 19:42:28 dist-test-slave-07c2 krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:42:28.266870 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:28.248979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41092 (local address 127.14.16.67:41939)
0807 19:42:28.249144 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:28.249148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:28.249207 (+ 59us) server_negotiation.cc:407] Connection header received
0807 19:42:28.249422 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:28.249426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:28.249488 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:28.249574 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:28.251552 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:28.252456 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:28.254423 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:28.254899 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:28.254998 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:28.260700 (+ 5702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:28.260712 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:28.260731 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:28.260778 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:28.263941 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:28.264673 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:28.264681 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:28.264683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:28.264844 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:28.265233 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:28.265239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:28.265244 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:28.265604 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:28.265972 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:28.266236 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 19:42:28.266464 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:42:28.267620 1291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:28.392458 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 1164
I20260807 19:42:28.415360 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 795
I20260807 19:42:28.416420 859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:28.527992 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 795
2026-08-07T19:42:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4535 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T19:42:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:28Z Disabled control of system clock
I20260807 19:42:28.626724 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:46871
--webserver_interface=127.14.16.126
--webserver_port=0
--builtin_ntp_servers=127.14.16.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:46871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:42:28.969370 1349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:28.969666 1349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:28.969750 1349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:28.977218 1349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:28.977341 1349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:28.977382 1349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:28.977458 1349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:28.977504 1349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:28.987231 1349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:46871
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:46871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:28.988531 1349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:28.990350 1349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:29.002790 1354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:29.002755 1355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:29.002727 1357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:29.003122 1349 server_base.cc:1034] running on GCE node
I20260807 19:42:29.004307 1349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:29.005791 1349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:29.007035 1349 hybrid_clock.cc:648] HybridClock initialized: now 1786131749006981 us; error 49 us; skew 500 ppm
I20260807 19:42:29.007476 1349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:29.009948 1349 webserver.cc:460] Webserver started at http://127.14.16.126:46439/ using document root <none> and password file <none>
I20260807 19:42:29.010677 1349 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:29.010818 1349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:29.011159 1349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:29.014178 1349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "5ec3e132e76b4f30a4f3d88adada16b8"
format_stamp: "Formatted at 2026-08-07 19:42:29 on dist-test-slave-07c2"
server_key: "550dc2b2632b7684c1d9a35ae08a19ae"
server_key_iv: "a9c35c00dd57e83f682cfbe895831c60"
server_key_version: "encryptionkey@0"
I20260807 19:42:29.015025 1349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "5ec3e132e76b4f30a4f3d88adada16b8"
format_stamp: "Formatted at 2026-08-07 19:42:29 on dist-test-slave-07c2"
server_key: "550dc2b2632b7684c1d9a35ae08a19ae"
server_key_iv: "a9c35c00dd57e83f682cfbe895831c60"
server_key_version: "encryptionkey@0"
I20260807 19:42:29.020378 1349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:42:29.023881 1364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:29.025205 1349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 19:42:29.025393 1349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "5ec3e132e76b4f30a4f3d88adada16b8"
format_stamp: "Formatted at 2026-08-07 19:42:29 on dist-test-slave-07c2"
server_key: "550dc2b2632b7684c1d9a35ae08a19ae"
server_key_iv: "a9c35c00dd57e83f682cfbe895831c60"
server_key_version: "encryptionkey@0"
I20260807 19:42:29.025615 1349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:29.069680 1349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:29.070564 1349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:29.070874 1349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:29.090229 1349 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:46871
I20260807 19:42:29.090247 1418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:46871 every 8 connection(s)
I20260807 19:42:29.091656 1349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:29.095485 1419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:29.096943 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 1349
I20260807 19:42:29.097157 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:29.097760 14401 external_mini_cluster.cc:1428] Setting key 7f27e89849015caeebf38970caa03384
I20260807 19:42:29.109903 1419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8: Bootstrap starting.
I20260807 19:42:29.111339 1422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:29.099599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35658 (local address 127.14.16.126:46871)
0807 19:42:29.100630 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:29.100643 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:29.100688 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:42:29.102356 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:29.102375 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:29.102831 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:29.103153 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:29.103776 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:29.105018 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:29.106965 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:29.107696 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:29.107832 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:29.109067 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:29.109074 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:29.109088 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:29.109595 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:29.109679 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:29.109700 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:29.109799 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:29.110054 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 19:42:29.110393 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":373,"threads_started":1}
I20260807 19:42:29.113485 1419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:29.114718 1419 log.cc:824] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:29.117296 1419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8: No bootstrap required, opened a new log
I20260807 19:42:29.124727 1419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ec3e132e76b4f30a4f3d88adada16b8" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46871 } }
I20260807 19:42:29.125119 1419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:29.125237 1419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec3e132e76b4f30a4f3d88adada16b8, State: Initialized, Role: FOLLOWER
I20260807 19:42:29.125916 1419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ec3e132e76b4f30a4f3d88adada16b8" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46871 } }
I20260807 19:42:29.126205 1419 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:29.126359 1419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:29.126538 1419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:29.128880 1419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ec3e132e76b4f30a4f3d88adada16b8" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46871 } }
I20260807 19:42:29.129464 1419 leader_election.cc:302] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ec3e132e76b4f30a4f3d88adada16b8; no voters:
I20260807 19:42:29.130198 1419 leader_election.cc:288] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:29.130295 1424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:29.131211 1424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [term 1 LEADER]: Becoming Leader. State: Replica: 5ec3e132e76b4f30a4f3d88adada16b8, State: Running, Role: LEADER
I20260807 19:42:29.131793 1424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ec3e132e76b4f30a4f3d88adada16b8" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46871 } }
I20260807 19:42:29.132504 1419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:29.134522 1426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ec3e132e76b4f30a4f3d88adada16b8. Latest consensus state: current_term: 1 leader_uuid: "5ec3e132e76b4f30a4f3d88adada16b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ec3e132e76b4f30a4f3d88adada16b8" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46871 } } }
I20260807 19:42:29.134994 1426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:29.135492 1425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ec3e132e76b4f30a4f3d88adada16b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ec3e132e76b4f30a4f3d88adada16b8" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46871 } } }
I20260807 19:42:29.135834 1425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:29.139678 1432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:29.144239 1432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:29.152541 1432 catalog_manager.cc:1347] Generated new cluster ID: 87d584057e01454ea44912a2ba8ba12a
I20260807 19:42:29.152664 1432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:29.176064 1432 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:29.177836 1432 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:29.197782 1432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8: Generated new TSK 0
I20260807 19:42:29.198561 1432 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:29.208017 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:46871
--builtin_ntp_servers=127.14.16.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:42:29.550304 1443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:29.550617 1443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:29.550709 1443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:29.558230 1443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:29.558344 1443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:29.558480 1443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:42:29.568424 1443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:46871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:29.569733 1443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:29.571511 1443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:29.586745 1448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:29.587117 1449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:29.589624 1452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:29.589788 1443 server_base.cc:1034] running on GCE node
I20260807 19:42:29.590546 1443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:29.591583 1443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:29.592845 1443 hybrid_clock.cc:648] HybridClock initialized: now 1786131749592774 us; error 63 us; skew 500 ppm
I20260807 19:42:29.593207 1443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:29.595708 1443 webserver.cc:460] Webserver started at http://127.14.16.65:42235/ using document root <none> and password file <none>
I20260807 19:42:29.596442 1443 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:29.596580 1443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:29.596926 1443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:29.600060 1443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "14aa9f7d361345c8890176770d5dfd9a"
format_stamp: "Formatted at 2026-08-07 19:42:29 on dist-test-slave-07c2"
server_key: "6ed09278129101909b0766f0e4573c51"
server_key_iv: "4b97188c22067554ffcbaf5f64e6c82e"
server_key_version: "encryptionkey@0"
I20260807 19:42:29.600888 1443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "14aa9f7d361345c8890176770d5dfd9a"
format_stamp: "Formatted at 2026-08-07 19:42:29 on dist-test-slave-07c2"
server_key: "6ed09278129101909b0766f0e4573c51"
server_key_iv: "4b97188c22067554ffcbaf5f64e6c82e"
server_key_version: "encryptionkey@0"
I20260807 19:42:29.606397 1443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 19:42:29.610230 1457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:29.611739 1443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:29.611925 1443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "14aa9f7d361345c8890176770d5dfd9a"
format_stamp: "Formatted at 2026-08-07 19:42:29 on dist-test-slave-07c2"
server_key: "6ed09278129101909b0766f0e4573c51"
server_key_iv: "4b97188c22067554ffcbaf5f64e6c82e"
server_key_version: "encryptionkey@0"
I20260807 19:42:29.612166 1443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:29.630013 1443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:29.630925 1443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:29.631242 1443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:29.632388 1443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:29.633864 1443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:29.633981 1443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:29.634117 1443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:29.634220 1443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:29.667522 1443 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:39209
I20260807 19:42:29.667600 1573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:39209 every 8 connection(s)
I20260807 19:42:29.668934 1443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:29.669592 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 1443
I20260807 19:42:29.669791 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:29.670408 14401 external_mini_cluster.cc:1428] Setting key 44fab85238bb2bbab12d4cdace7d167b
I20260807 19:42:29.673816 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:46871
--builtin_ntp_servers=127.14.16.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:42:29.683555 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:29.671839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:48577 (local address 127.14.16.126:46871)
0807 19:42:29.672339 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:29.672346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:29.673528 (+ 1182us) server_negotiation.cc:407] Connection header received
0807 19:42:29.675373 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:29.675378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:29.675467 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:29.675583 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:29.676651 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:29.677662 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:29.680128 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:29.680743 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:29.680862 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:29.682594 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:29.682602 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:29.682609 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:29.682631 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:29.682686 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:29.682708 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:29.682791 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:29.683036 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 19:42:29.683138 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":253,"threads_started":1}
I20260807 19:42:29.684732 1577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:29.672451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:46871 (local address 127.14.16.65:48577)
0807 19:42:29.673382 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:29.673415 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:29.675087 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:29.675831 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:29.675844 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:29.675939 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:29.676491 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:29.676508 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:29.677846 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:29.677854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:29.679981 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:29.680001 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:29.681077 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:29.681089 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:29.681428 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:29.682242 (+ 814us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:29.682247 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:29.682299 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:29.682400 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:29.682813 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:29.682827 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:29.682837 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 19:42:29.683014 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 19:42:29.683432 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":335,"threads_started":1}
I20260807 19:42:29.686275 1574 heartbeater.cc:347] Connected to a master server at 127.14.16.126:46871
I20260807 19:42:29.686604 1574 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:29.687441 1574 heartbeater.cc:511] Master 127.14.16.126:46871 requested a full tablet report, sending...
I20260807 19:42:29.689392 1382 ts_manager.cc:194] Registered new tserver with Master: 14aa9f7d361345c8890176770d5dfd9a (127.14.16.65:39209)
I20260807 19:42:29.690840 1382 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.16.65:48577
W20260807 19:42:30.013013 1578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:30.013309 1578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:30.013487 1578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:30.020776 1578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:30.020879 1578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:30.021095 1578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:42:30.030212 1578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:46871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:30.031687 1578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:30.033591 1578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:30.047209 1583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:30.047343 1584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:30.047317 1586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:30.048298 1578 server_base.cc:1034] running on GCE node
I20260807 19:42:30.048946 1578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:30.049960 1578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:30.051182 1578 hybrid_clock.cc:648] HybridClock initialized: now 1786131750051136 us; error 32 us; skew 500 ppm
I20260807 19:42:30.051522 1578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:30.053871 1578 webserver.cc:460] Webserver started at http://127.14.16.66:41241/ using document root <none> and password file <none>
I20260807 19:42:30.054711 1578 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:30.054880 1578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:30.055235 1578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:30.058117 1578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "e3c86b86a80a4768a6032e1960fc7696"
format_stamp: "Formatted at 2026-08-07 19:42:30 on dist-test-slave-07c2"
server_key: "06d06e785b9f01a36f23834267d5e0a1"
server_key_iv: "6d1a98e8d1e1416cc3995d924e9dd535"
server_key_version: "encryptionkey@0"
I20260807 19:42:30.058893 1578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "e3c86b86a80a4768a6032e1960fc7696"
format_stamp: "Formatted at 2026-08-07 19:42:30 on dist-test-slave-07c2"
server_key: "06d06e785b9f01a36f23834267d5e0a1"
server_key_iv: "6d1a98e8d1e1416cc3995d924e9dd535"
server_key_version: "encryptionkey@0"
I20260807 19:42:30.063936 1578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 19:42:30.067353 1592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:30.068734 1578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:30.068902 1578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "e3c86b86a80a4768a6032e1960fc7696"
format_stamp: "Formatted at 2026-08-07 19:42:30 on dist-test-slave-07c2"
server_key: "06d06e785b9f01a36f23834267d5e0a1"
server_key_iv: "6d1a98e8d1e1416cc3995d924e9dd535"
server_key_version: "encryptionkey@0"
I20260807 19:42:30.069129 1578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:30.089617 1578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:30.090471 1578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:30.090795 1578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:30.091917 1578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:30.093384 1578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:30.093505 1578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:30.093642 1578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:30.093732 1578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:30.128123 1578 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:44615
I20260807 19:42:30.128212 1709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:44615 every 8 connection(s)
I20260807 19:42:30.129489 1578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:30.137003 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 1578
I20260807 19:42:30.137208 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:30.137673 14401 external_mini_cluster.cc:1428] Setting key 2cfa445271b52b894509a9684dffca8b
I20260807 19:42:30.140762 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:46871
--builtin_ntp_servers=127.14.16.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:42:30.142844 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.132133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:51039 (local address 127.14.16.126:46871)
0807 19:42:30.132343 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:30.132347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:30.133872 (+ 1525us) server_negotiation.cc:407] Connection header received
0807 19:42:30.135730 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:30.135736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:30.135825 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:30.135934 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:30.137172 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.137976 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.140007 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.140471 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.140602 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.142045 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:30.142049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:30.142053 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:30.142070 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:30.142100 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:30.142112 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:30.142209 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:30.142442 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 19:42:30.142540 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:42:30.143895 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.132648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:46871 (local address 127.14.16.66:51039)
0807 19:42:30.133704 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:30.133756 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:30.135387 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:30.136160 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:30.136172 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:30.136295 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:30.137008 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.137024 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.138127 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.138132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.139871 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.139883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.140684 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.140691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.140973 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.141720 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:30.141723 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:30.141776 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:30.141852 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:30.142211 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:30.142224 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:30.142234 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 19:42:30.142426 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 19:42:30.142811 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":355,"threads_started":1}
I20260807 19:42:30.145328 1710 heartbeater.cc:347] Connected to a master server at 127.14.16.126:46871
I20260807 19:42:30.145630 1710 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:30.146328 1710 heartbeater.cc:511] Master 127.14.16.126:46871 requested a full tablet report, sending...
I20260807 19:42:30.147709 1382 ts_manager.cc:194] Registered new tserver with Master: e3c86b86a80a4768a6032e1960fc7696 (127.14.16.66:44615)
I20260807 19:42:30.148502 1382 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.16.66:51039
W20260807 19:42:30.479391 1713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:30.479668 1713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:30.479758 1713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:30.487053 1713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:30.487164 1713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:30.487269 1713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:42:30.496361 1713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:46871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:30.497596 1713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:30.499279 1713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:30.512420 1722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:30.512442 1720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:30.513139 1713 server_base.cc:1034] running on GCE node
W20260807 19:42:30.513190 1718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:30.513851 1713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:30.514803 1713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:30.516034 1713 hybrid_clock.cc:648] HybridClock initialized: now 1786131750515998 us; error 49 us; skew 500 ppm
I20260807 19:42:30.516362 1713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:30.518697 1713 webserver.cc:460] Webserver started at http://127.14.16.67:43313/ using document root <none> and password file <none>
I20260807 19:42:30.519469 1713 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:30.519609 1713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:30.519933 1713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:30.522847 1713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "0c28aa73ab1545e1bd2a0ae715332694"
format_stamp: "Formatted at 2026-08-07 19:42:30 on dist-test-slave-07c2"
server_key: "fa0fcb76d235113a542b434a894d0d66"
server_key_iv: "160ccaec9de064896abf7b3035cb5c61"
server_key_version: "encryptionkey@0"
I20260807 19:42:30.523699 1713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "0c28aa73ab1545e1bd2a0ae715332694"
format_stamp: "Formatted at 2026-08-07 19:42:30 on dist-test-slave-07c2"
server_key: "fa0fcb76d235113a542b434a894d0d66"
server_key_iv: "160ccaec9de064896abf7b3035cb5c61"
server_key_version: "encryptionkey@0"
I20260807 19:42:30.528761 1713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 19:42:30.532167 1728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:30.533658 1713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:42:30.533991 1713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "0c28aa73ab1545e1bd2a0ae715332694"
format_stamp: "Formatted at 2026-08-07 19:42:30 on dist-test-slave-07c2"
server_key: "fa0fcb76d235113a542b434a894d0d66"
server_key_iv: "160ccaec9de064896abf7b3035cb5c61"
server_key_version: "encryptionkey@0"
I20260807 19:42:30.534232 1713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:30.552310 1713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:30.553156 1713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:30.553484 1713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:30.554461 1713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:30.555945 1713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:30.556077 1713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:30.556214 1713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:30.556306 1713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:30.590412 1713 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:34499
I20260807 19:42:30.590471 1845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:34499 every 8 connection(s)
I20260807 19:42:30.591830 1713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:30.592276 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 1713
I20260807 19:42:30.592468 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:30.593101 14401 external_mini_cluster.cc:1428] Setting key d025e15cf81f3b107e016960a367274c
I20260807 19:42:30.605098 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.594696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:54019 (local address 127.14.16.126:46871)
0807 19:42:30.594884 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:30.594888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:30.596434 (+ 1546us) server_negotiation.cc:407] Connection header received
0807 19:42:30.598507 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:30.598515 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:30.598704 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:30.598856 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:30.599977 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.600723 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.602266 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.602711 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.602843 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.604256 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:30.604262 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:30.604267 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:30.604293 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:30.604323 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:30.604335 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:30.604432 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:30.604666 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 19:42:30.604773 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:42:30.606081 1848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.595343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:46871 (local address 127.14.16.67:54019)
0807 19:42:30.596293 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:30.596330 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:30.598223 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:30.599085 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:30.599093 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:30.599185 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:30.599810 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.599827 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.600889 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.600893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.602112 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.602124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.602932 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.602939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.603175 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.603936 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:30.603939 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:30.603976 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:30.604052 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:30.604431 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:30.604444 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:30.604454 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 19:42:30.604636 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 19:42:30.605004 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":298,"threads_started":1}
I20260807 19:42:30.607600 1846 heartbeater.cc:347] Connected to a master server at 127.14.16.126:46871
I20260807 19:42:30.607919 1846 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:30.608846 1846 heartbeater.cc:511] Master 127.14.16.126:46871 requested a full tablet report, sending...
I20260807 19:42:30.610199 1382 ts_manager.cc:194] Registered new tserver with Master: 0c28aa73ab1545e1bd2a0ae715332694 (127.14.16.67:34499)
I20260807 19:42:30.610961 1382 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.16.67:54019
I20260807 19:42:30.620926 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:42:30.631247 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.625047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35660 (local address 127.14.16.126:46871)
0807 19:42:30.625263 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:30.625271 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:30.625730 (+ 459us) server_negotiation.cc:407] Connection header received
0807 19:42:30.625875 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:30.625878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:30.625954 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:30.626038 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:30.626510 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.627244 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.628706 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.629121 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.629266 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.630404 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:30.630414 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:30.630418 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:30.630437 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:30.630469 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:30.630482 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:30.630637 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:30.630786 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 19:42:30.630859 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:42:30.635528 1382 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:30.642696 1382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:30.677913 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.654897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.14.16.67:34499)
0807 19:42:30.667968 (+ 13071us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:30.667981 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:30.668030 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:42:30.668190 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:30.668202 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:30.668530 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:30.668790 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:30.670250 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.671987 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.675609 (+ 3622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.676578 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.676740 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.677052 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:30.677377 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 19:42:30.677559 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12819,"thread_start_us":13721,"threads_started":1}
I20260807 19:42:30.678086 1863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.654526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:34499 (local address 127.0.0.1:48380)
0807 19:42:30.664352 (+ 9826us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:30.664417 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:30.664645 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:30.669204 (+ 4559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:30.669217 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:30.669263 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:30.669844 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.669861 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.672173 (+ 2312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.672182 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.675381 (+ 3199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.675399 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.676831 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.676838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.677127 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.677184 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 19:42:30.677334 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 19:42:30.677529 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9452,"mutex_wait_us":60,"thread_start_us":16166,"threads_started":1}
I20260807 19:42:30.681164 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.655117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:39209 (local address 127.0.0.1:54332)
0807 19:42:30.670851 (+ 15734us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:30.670868 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:30.671053 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:30.671921 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:30.671926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:30.671970 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:30.673044 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.673061 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.675687 (+ 2626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.675692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.677885 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.677896 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.680133 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.680138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.680422 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.680565 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 19:42:30.680662 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 19:42:30.680796 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15614,"mutex_wait_us":153,"spinlock_wait_cycles":8320,"thread_start_us":8677,"threads_started":1}
I20260807 19:42:30.681315 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.655729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44198 (local address 127.14.16.66:44615)
0807 19:42:30.669238 (+ 13509us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:30.669246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:30.670667 (+ 1421us) server_negotiation.cc:407] Connection header received
0807 19:42:30.671154 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:30.671164 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:30.671424 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:30.671596 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:30.673531 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.675103 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.678116 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.679595 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.679730 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.679961 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:30.680748 (+ 787us) server_negotiation.cc:299] Negotiation successful
0807 19:42:30.680927 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13346,"thread_start_us":14125,"threads_started":1}
I20260807 19:42:30.681303 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.656312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54332 (local address 127.14.16.65:39209)
0807 19:42:30.669655 (+ 13343us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:30.669674 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:30.670961 (+ 1287us) server_negotiation.cc:407] Connection header received
0807 19:42:30.671234 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:30.671241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:30.671441 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:30.671604 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:30.673924 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.675497 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.678123 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.679602 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.679731 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.679991 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:30.680730 (+ 739us) server_negotiation.cc:299] Negotiation successful
0807 19:42:30.680889 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13182,"thread_start_us":13380,"threads_started":1}
I20260807 19:42:30.682147 1861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.654803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:44615 (local address 127.0.0.1:44198)
0807 19:42:30.670521 (+ 15718us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:30.670543 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:30.670948 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:30.671921 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:30.671926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:30.671963 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:30.673299 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.673311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.675327 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.675332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.677795 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.677812 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.680128 (+ 2316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.680134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.680425 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.680565 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 19:42:30.681789 (+ 1224us) client_negotiation.cc:240] Negotiation successful
0807 19:42:30.681885 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15555,"mutex_wait_us":332,"spinlock_wait_cycles":896,"thread_start_us":15416,"threads_started":1}
I20260807 19:42:30.683163 1778 tablet_service.cc:1467] Processing CreateTablet for tablet 0b769af863a74ea9804e1508789eb75f (DEFAULT_TABLE table=test-table [id=a66622eb70d448ed92773c112e52ac72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:30.684511 1778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b769af863a74ea9804e1508789eb75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:30.685909 1507 tablet_service.cc:1467] Processing CreateTablet for tablet 0b769af863a74ea9804e1508789eb75f (DEFAULT_TABLE table=test-table [id=a66622eb70d448ed92773c112e52ac72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:30.685909 1640 tablet_service.cc:1467] Processing CreateTablet for tablet 0b769af863a74ea9804e1508789eb75f (DEFAULT_TABLE table=test-table [id=a66622eb70d448ed92773c112e52ac72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:30.687275 1507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b769af863a74ea9804e1508789eb75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:30.687278 1640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b769af863a74ea9804e1508789eb75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:30.694448 1574 heartbeater.cc:503] Master 127.14.16.126:46871 was elected leader, sending a full tablet report...
I20260807 19:42:30.697973 1866 tablet_bootstrap.cc:492] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694: Bootstrap starting.
I20260807 19:42:30.701877 1866 tablet_bootstrap.cc:654] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:30.702581 1867 tablet_bootstrap.cc:492] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a: Bootstrap starting.
I20260807 19:42:30.703445 1866 log.cc:824] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:30.706606 1867 tablet_bootstrap.cc:654] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:30.707235 1866 tablet_bootstrap.cc:492] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694: No bootstrap required, opened a new log
I20260807 19:42:30.707643 1866 ts_tablet_manager.cc:1397] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260807 19:42:30.707958 1867 log.cc:824] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:30.708755 1868 tablet_bootstrap.cc:492] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696: Bootstrap starting.
I20260807 19:42:30.710557 1867 tablet_bootstrap.cc:492] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a: No bootstrap required, opened a new log
I20260807 19:42:30.710839 1867 ts_tablet_manager.cc:1397] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:42:30.712411 1868 tablet_bootstrap.cc:654] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:30.713603 1868 log.cc:824] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:30.716782 1868 tablet_bootstrap.cc:492] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696: No bootstrap required, opened a new log
I20260807 19:42:30.717053 1868 ts_tablet_manager.cc:1397] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.006s
I20260807 19:42:30.718205 1866 raft_consensus.cc:348] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c28aa73ab1545e1bd2a0ae715332694" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 34499 } } peers { permanent_uuid: "e3c86b86a80a4768a6032e1960fc7696" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44615 } } peers { permanent_uuid: "14aa9f7d361345c8890176770d5dfd9a" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39209 } }
I20260807 19:42:30.718677 1866 raft_consensus.cc:374] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:30.718829 1866 raft_consensus.cc:729] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c28aa73ab1545e1bd2a0ae715332694, State: Initialized, Role: FOLLOWER
I20260807 19:42:30.718659 1867 raft_consensus.cc:348] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c28aa73ab1545e1bd2a0ae715332694" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 34499 } } peers { permanent_uuid: "e3c86b86a80a4768a6032e1960fc7696" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44615 } } peers { permanent_uuid: "14aa9f7d361345c8890176770d5dfd9a" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39209 } }
I20260807 19:42:30.719121 1867 raft_consensus.cc:374] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:30.719245 1867 raft_consensus.cc:729] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14aa9f7d361345c8890176770d5dfd9a, State: Initialized, Role: FOLLOWER
I20260807 19:42:30.719494 1866 consensus_queue.cc:260] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c28aa73ab1545e1bd2a0ae715332694" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 34499 } } peers { permanent_uuid: "e3c86b86a80a4768a6032e1960fc7696" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44615 } } peers { permanent_uuid: "14aa9f7d361345c8890176770d5dfd9a" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39209 } }
I20260807 19:42:30.719823 1867 consensus_queue.cc:260] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c28aa73ab1545e1bd2a0ae715332694" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 34499 } } peers { permanent_uuid: "e3c86b86a80a4768a6032e1960fc7696" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44615 } } peers { permanent_uuid: "14aa9f7d361345c8890176770d5dfd9a" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39209 } }
I20260807 19:42:30.722851 1867 ts_tablet_manager.cc:1428] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260807 19:42:30.726843 1846 heartbeater.cc:503] Master 127.14.16.126:46871 was elected leader, sending a full tablet report...
I20260807 19:42:30.727792 1866 ts_tablet_manager.cc:1428] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694: Time spent starting tablet: real 0.020s user 0.010s sys 0.009s
I20260807 19:42:30.729645 1868 raft_consensus.cc:348] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c28aa73ab1545e1bd2a0ae715332694" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 34499 } } peers { permanent_uuid: "e3c86b86a80a4768a6032e1960fc7696" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44615 } } peers { permanent_uuid: "14aa9f7d361345c8890176770d5dfd9a" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39209 } }
I20260807 19:42:30.730046 1868 raft_consensus.cc:374] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:30.730170 1868 raft_consensus.cc:729] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c86b86a80a4768a6032e1960fc7696, State: Initialized, Role: FOLLOWER
I20260807 19:42:30.730814 1868 consensus_queue.cc:260] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c28aa73ab1545e1bd2a0ae715332694" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 34499 } } peers { permanent_uuid: "e3c86b86a80a4768a6032e1960fc7696" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44615 } } peers { permanent_uuid: "14aa9f7d361345c8890176770d5dfd9a" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39209 } }
I20260807 19:42:30.732072 1710 heartbeater.cc:503] Master 127.14.16.126:46871 was elected leader, sending a full tablet report...
I20260807 19:42:30.732686 1868 ts_tablet_manager.cc:1428] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
W20260807 19:42:30.843866 1847 tablet.cc:2369] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:42:30.881631 1711 tablet.cc:2369] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:30.889773 1873 raft_consensus.cc:482] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:30.890130 1873 raft_consensus.cc:504] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c28aa73ab1545e1bd2a0ae715332694" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 34499 } } peers { permanent_uuid: "e3c86b86a80a4768a6032e1960fc7696" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44615 } } peers { permanent_uuid: "14aa9f7d361345c8890176770d5dfd9a" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39209 } }
I20260807 19:42:30.891997 1873 leader_election.cc:288] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c28aa73ab1545e1bd2a0ae715332694 (127.14.16.67:34499), e3c86b86a80a4768a6032e1960fc7696 (127.14.16.66:44615)
I20260807 19:42:30.899129 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.892471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:45523 (local address 127.14.16.66:44615)
0807 19:42:30.892766 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:30.892775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:30.893214 (+ 439us) server_negotiation.cc:407] Connection header received
0807 19:42:30.893442 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:30.893448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:30.893546 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:30.893652 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:30.894484 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.895467 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.897430 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.898168 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.898292 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.898385 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:30.898762 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 19:42:30.898881 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:42:30.899140 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.892471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:60573 (local address 127.14.16.67:34499)
0807 19:42:30.892696 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:30.892704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:30.893337 (+ 633us) server_negotiation.cc:407] Connection header received
0807 19:42:30.893470 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:30.893475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:30.893549 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:30.893652 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:30.894446 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.895409 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.897477 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.898245 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.898408 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.898484 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:30.898757 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 19:42:30.898885 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:42:30.899251 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.892345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:34499 (local address 127.14.16.65:60573)
0807 19:42:30.893223 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:30.893239 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:30.893362 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:30.893683 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:30.893688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:30.893726 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:30.894226 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.894246 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.895554 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.895562 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.897233 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.897246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.898451 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.898456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.898586 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.898620 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:42:30.898718 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 19:42:30.898856 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":116,"thread_start_us":500,"threads_started":1}
I20260807 19:42:30.899326 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.892345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:44615 (local address 127.14.16.65:45523)
0807 19:42:30.893086 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:30.893107 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:30.893282 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:30.893683 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:30.893688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:30.893791 (+ 103us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:30.894310 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.894321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.895619 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.895624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.897220 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.897233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.898354 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.898360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.898550 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.898615 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 19:42:30.898732 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 19:42:30.898859 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":123,"spinlock_wait_cycles":23808,"thread_start_us":431,"threads_started":1}
I20260807 19:42:30.900331 1798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b769af863a74ea9804e1508789eb75f" candidate_uuid: "14aa9f7d361345c8890176770d5dfd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c28aa73ab1545e1bd2a0ae715332694" is_pre_election: true
I20260807 19:42:30.900543 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b769af863a74ea9804e1508789eb75f" candidate_uuid: "14aa9f7d361345c8890176770d5dfd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c86b86a80a4768a6032e1960fc7696" is_pre_election: true
I20260807 19:42:30.900957 1798 raft_consensus.cc:2454] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14aa9f7d361345c8890176770d5dfd9a in term 0.
I20260807 19:42:30.901091 1661 raft_consensus.cc:2454] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14aa9f7d361345c8890176770d5dfd9a in term 0.
I20260807 19:42:30.901741 1458 leader_election.cc:302] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0c28aa73ab1545e1bd2a0ae715332694, 14aa9f7d361345c8890176770d5dfd9a, e3c86b86a80a4768a6032e1960fc7696; no voters:
I20260807 19:42:30.902195 1873 raft_consensus.cc:2781] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:30.902355 1873 raft_consensus.cc:482] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:30.902477 1873 raft_consensus.cc:3037] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.904735 1873 raft_consensus.cc:504] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c28aa73ab1545e1bd2a0ae715332694" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 34499 } } peers { permanent_uuid: "e3c86b86a80a4768a6032e1960fc7696" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44615 } } peers { permanent_uuid: "14aa9f7d361345c8890176770d5dfd9a" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39209 } }
I20260807 19:42:30.905344 1873 leader_election.cc:288] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [CANDIDATE]: Term 1 election: Requested vote from peers 0c28aa73ab1545e1bd2a0ae715332694 (127.14.16.67:34499), e3c86b86a80a4768a6032e1960fc7696 (127.14.16.66:44615)
I20260807 19:42:30.905851 1798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b769af863a74ea9804e1508789eb75f" candidate_uuid: "14aa9f7d361345c8890176770d5dfd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c28aa73ab1545e1bd2a0ae715332694"
I20260807 19:42:30.905857 1661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b769af863a74ea9804e1508789eb75f" candidate_uuid: "14aa9f7d361345c8890176770d5dfd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c86b86a80a4768a6032e1960fc7696"
I20260807 19:42:30.906059 1798 raft_consensus.cc:3037] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.906067 1661 raft_consensus.cc:3037] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.908339 1798 raft_consensus.cc:2454] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14aa9f7d361345c8890176770d5dfd9a in term 1.
I20260807 19:42:30.908603 1661 raft_consensus.cc:2454] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14aa9f7d361345c8890176770d5dfd9a in term 1.
I20260807 19:42:30.908785 1460 leader_election.cc:302] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c28aa73ab1545e1bd2a0ae715332694, 14aa9f7d361345c8890176770d5dfd9a; no voters:
I20260807 19:42:30.909091 1873 raft_consensus.cc:2781] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:30.909545 1873 raft_consensus.cc:686] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 1 LEADER]: Becoming Leader. State: Replica: 14aa9f7d361345c8890176770d5dfd9a, State: Running, Role: LEADER
I20260807 19:42:30.910008 1873 consensus_queue.cc:237] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c28aa73ab1545e1bd2a0ae715332694" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 34499 } } peers { permanent_uuid: "e3c86b86a80a4768a6032e1960fc7696" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44615 } } peers { permanent_uuid: "14aa9f7d361345c8890176770d5dfd9a" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39209 } }
I20260807 19:42:30.914763 1381 catalog_manager.cc:5495] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 14aa9f7d361345c8890176770d5dfd9a (127.14.16.65). New cstate: current_term: 1 leader_uuid: "14aa9f7d361345c8890176770d5dfd9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c28aa73ab1545e1bd2a0ae715332694" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 34499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3c86b86a80a4768a6032e1960fc7696" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14aa9f7d361345c8890176770d5dfd9a" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 39209 } health_report { overall_health: HEALTHY } } }
W20260807 19:42:30.921324 1575 tablet.cc:2369] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:30.964457 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.958904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54344 (local address 127.14.16.65:39209)
0807 19:42:30.959101 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:30.959105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:30.959209 (+ 104us) server_negotiation.cc:407] Connection header received
0807 19:42:30.959440 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:30.959443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:30.959580 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:30.959764 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:30.960227 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.961152 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.962474 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.962881 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.962972 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.963486 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:30.963994 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:30.964123 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:30.964231 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 19:42:30.964268 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:42:30.972297 1661 raft_consensus.cc:1261] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696 [term 1 FOLLOWER]: Refusing update from remote peer 14aa9f7d361345c8890176770d5dfd9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:30.972319 1798 raft_consensus.cc:1261] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694 [term 1 FOLLOWER]: Refusing update from remote peer 14aa9f7d361345c8890176770d5dfd9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:30.973287 1877 consensus_queue.cc:1035] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c28aa73ab1545e1bd2a0ae715332694" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 34499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:30.973621 1873 consensus_queue.cc:1035] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c86b86a80a4768a6032e1960fc7696" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 44615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:30.983440 1880 mvcc.cc:205] Tried to move back new op lower bound from 7315995651973185536 to 7315995651731042304. Current Snapshot: MvccSnapshot[applied={T|T < 7315995651973185536}]
I20260807 19:42:30.987792 1882 mvcc.cc:205] Tried to move back new op lower bound from 7315995651973185536 to 7315995651731042304. Current Snapshot: MvccSnapshot[applied={T|T < 7315995651973185536}]
I20260807 19:42:30.988235 1881 mvcc.cc:205] Tried to move back new op lower bound from 7315995651973185536 to 7315995651731042304. Current Snapshot: MvccSnapshot[applied={T|T < 7315995651973185536}]
I20260807 19:42:30.997462 1382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:30.997776 1382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:31.001408 1382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ec3e132e76b4f30a4f3d88adada16b8: Sending DeleteTablet for 3 replicas of tablet 0b769af863a74ea9804e1508789eb75f
I20260807 19:42:31.002511 1778 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b769af863a74ea9804e1508789eb75f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:30 UTC) from {username='slave'} at 127.0.0.1:48380
I20260807 19:42:31.002517 1640 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b769af863a74ea9804e1508789eb75f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:30 UTC) from {username='slave'} at 127.0.0.1:44198
I20260807 19:42:31.003214 1507 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b769af863a74ea9804e1508789eb75f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:30 UTC) from {username='slave'} at 127.0.0.1:54332
I20260807 19:42:31.003670 1889 tablet_replica.cc:330] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694: stopping tablet replica
I20260807 19:42:31.004115 1889 raft_consensus.cc:2229] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:31.004320 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 1443
I20260807 19:42:31.004608 1889 raft_consensus.cc:2258] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:31.005364 1891 tablet_replica.cc:330] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a: stopping tablet replica
I20260807 19:42:31.005729 1891 raft_consensus.cc:2229] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:31.006345 1891 raft_consensus.cc:2258] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:31.007036 1889 ts_tablet_manager.cc:1905] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:31.008955 1891 ts_tablet_manager.cc:1905] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:31.014305 1889 ts_tablet_manager.cc:1918] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:31.014569 1889 log.cc:1196] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/0b769af863a74ea9804e1508789eb75f
I20260807 19:42:31.015463 1889 ts_tablet_manager.cc:1939] T 0b769af863a74ea9804e1508789eb75f P 0c28aa73ab1545e1bd2a0ae715332694: Deleting consensus metadata
I20260807 19:42:31.016678 1891 ts_tablet_manager.cc:1918] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:31.016944 1891 log.cc:1196] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/0b769af863a74ea9804e1508789eb75f
I20260807 19:42:31.017726 1891 ts_tablet_manager.cc:1939] T 0b769af863a74ea9804e1508789eb75f P 14aa9f7d361345c8890176770d5dfd9a: Deleting consensus metadata
I20260807 19:42:31.017661 1367 catalog_manager.cc:4841] TS 0c28aa73ab1545e1bd2a0ae715332694 (127.14.16.67:34499): tablet 0b769af863a74ea9804e1508789eb75f (table test-table [id=a66622eb70d448ed92773c112e52ac72]) successfully deleted
I20260807 19:42:31.019240 1368 catalog_manager.cc:4841] TS 14aa9f7d361345c8890176770d5dfd9a (127.14.16.65:39209): tablet 0b769af863a74ea9804e1508789eb75f (table test-table [id=a66622eb70d448ed92773c112e52ac72]) successfully deleted
I20260807 19:42:31.020069 1890 tablet_replica.cc:330] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696: stopping tablet replica
I20260807 19:42:31.020574 1890 raft_consensus.cc:2229] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:31.021091 1890 raft_consensus.cc:2258] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:31.023178 1890 ts_tablet_manager.cc:1905] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:31.027714 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:31.006878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.14.16.65:39209)
0807 19:42:31.007229 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:31.007235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:31.019918 (+ 12683us) server_negotiation.cc:407] Connection header received
0807 19:42:31.020116 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:31.020123 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:31.020249 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:31.020377 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:31.020875 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.022092 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.023999 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.024648 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.024800 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:31.026020 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:31.026031 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:31.026048 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:31.026828 (+ 780us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:31.026894 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:31.026918 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:31.027027 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:31.027203 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 19:42:31.027328 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
I20260807 19:42:31.028512 1569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:31.029440 1890 ts_tablet_manager.cc:1918] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:31.029706 1890 log.cc:1196] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/0b769af863a74ea9804e1508789eb75f
I20260807 19:42:31.030220 1890 ts_tablet_manager.cc:1939] T 0b769af863a74ea9804e1508789eb75f P e3c86b86a80a4768a6032e1960fc7696: Deleting consensus metadata
I20260807 19:42:31.032007 1365 catalog_manager.cc:4841] TS e3c86b86a80a4768a6032e1960fc7696 (127.14.16.66:44615): tablet 0b769af863a74ea9804e1508789eb75f (table test-table [id=a66622eb70d448ed92773c112e52ac72]) successfully deleted
I20260807 19:42:31.127912 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 1443
I20260807 19:42:31.150734 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 1578
I20260807 19:42:31.157331 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:31.151544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44214 (local address 127.14.16.66:44615)
0807 19:42:31.151721 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:31.151726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:31.151744 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:42:31.151882 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:31.151886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:31.151977 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:31.152060 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:31.152446 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.153166 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.154429 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.154821 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.154912 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:31.155984 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:31.155993 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:31.156005 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:31.156579 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:31.156626 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:31.156651 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:31.156752 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:31.156871 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 19:42:31.156956 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:42:31.158020 1704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:31.249980 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 1578
I20260807 19:42:31.269208 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 1713
I20260807 19:42:31.276371 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:31.270085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48386 (local address 127.14.16.67:34499)
0807 19:42:31.270307 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:31.270313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:31.270335 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:42:31.270511 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:31.270517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:31.270615 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:31.270736 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:31.271125 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.271909 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.273190 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.273602 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.273734 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:31.274884 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:31.274894 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:31.274908 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:31.275596 (+ 688us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:31.275644 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:31.275663 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:31.275792 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:31.275924 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 19:42:31.276031 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:42:31.277040 1840 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:31.377264 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 1713
I20260807 19:42:31.399106 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 1349
I20260807 19:42:31.400297 1414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:31.511195 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 1349
2026-08-07T19:42:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2979 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:31 dist-test-slave-07c2 krb5kdc[1899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:31 dist-test-slave-07c2 krb5kdc[1899](info): set up 2 sockets
Aug 07 19:42:31 dist-test-slave-07c2 krb5kdc[1899](info): commencing operation
krb5kdc: starting...
W20260807 19:42:32.662767 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.009s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:32 dist-test-slave-07c2 krb5kdc[1899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:42:32.984649 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:43303
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:43303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:33.326799 1912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:33.327087 1912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:33.327234 1912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:33.334455 1912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:33.334565 1912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:33.334645 1912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:33.334780 1912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:33.334859 1912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:33.344834 1912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:43303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:43303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:33.346114 1912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:33.347431 1912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:33.360034 1918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:33.360108 1920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:33.360034 1917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:33.361297 1912 server_base.cc:1034] running on GCE node
I20260807 19:42:33.362052 1912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:33.363504 1912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:33.364727 1912 hybrid_clock.cc:648] HybridClock initialized: now 1786131753364692 us; error 51 us; skew 500 ppm
I20260807 19:42:33.365085 1912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:33 dist-test-slave-07c2 krb5kdc[1899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131753, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:33.372283 1912 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:42:33.373978 1912 webserver.cc:460] Webserver started at http://127.14.16.126:41689/ using document root <none> and password file <none>
I20260807 19:42:33.374691 1912 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:33.374845 1912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:33.375151 1912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:33.378219 1912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "d489b62c4b9849c0840567f2f7df2e53"
format_stamp: "Formatted at 2026-08-07 19:42:33 on dist-test-slave-07c2"
server_key: "2e07a348d4139df6a4635d560990687b"
server_key_iv: "f3d7c71236208a89caf9b3f5f229cd61"
server_key_version: "encryptionkey@0"
I20260807 19:42:33.378980 1912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "d489b62c4b9849c0840567f2f7df2e53"
format_stamp: "Formatted at 2026-08-07 19:42:33 on dist-test-slave-07c2"
server_key: "2e07a348d4139df6a4635d560990687b"
server_key_iv: "f3d7c71236208a89caf9b3f5f229cd61"
server_key_version: "encryptionkey@0"
I20260807 19:42:33.384323 1912 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 19:42:33.387883 1927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:33.389593 1912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:42:33.389801 1912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "d489b62c4b9849c0840567f2f7df2e53"
format_stamp: "Formatted at 2026-08-07 19:42:33 on dist-test-slave-07c2"
server_key: "2e07a348d4139df6a4635d560990687b"
server_key_iv: "f3d7c71236208a89caf9b3f5f229cd61"
server_key_version: "encryptionkey@0"
I20260807 19:42:33.390026 1912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:33.419726 1912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:33.424096 1912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:33.424465 1912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:33.443993 1912 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:43303
I20260807 19:42:33.444013 1979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:43303 every 8 connection(s)
I20260807 19:42:33.445423 1912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:33.449208 1980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:33.455369 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 1912
I20260807 19:42:33.455566 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:33.456084 14401 external_mini_cluster.cc:1428] Setting key 042d8962fe39b7dc8e49777c23ba4251
I20260807 19:42:33.460367 1980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53: Bootstrap starting.
I20260807 19:42:33.463635 1980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:33.464689 1980 log.cc:824] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:33.467937 1980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53: No bootstrap required, opened a new log
Aug 07 19:42:33 dist-test-slave-07c2 krb5kdc[1899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:42:33.476807 1980 raft_consensus.cc:348] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d489b62c4b9849c0840567f2f7df2e53" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43303 } }
I20260807 19:42:33.477349 1980 raft_consensus.cc:374] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:33.477492 1980 raft_consensus.cc:729] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d489b62c4b9849c0840567f2f7df2e53, State: Initialized, Role: FOLLOWER
I20260807 19:42:33.478215 1980 consensus_queue.cc:260] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d489b62c4b9849c0840567f2f7df2e53" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43303 } }
I20260807 19:42:33.478541 1980 raft_consensus.cc:388] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:33.478760 1980 raft_consensus.cc:482] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:33.478982 1980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:33.480494 1983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:33.458051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.14.16.126:43303)
0807 19:42:33.458831 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:33.458843 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:33.458936 (+ 93us) server_negotiation.cc:407] Connection header received
0807 19:42:33.460977 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:33.461001 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:33.461381 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:33.461775 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:33.463336 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:33.464698 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:33.466421 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:33.467088 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:33.467216 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:33.472141 (+ 4925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:33.472149 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:33.472165 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:33.472213 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:33.475656 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:33.476527 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:33.476534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:33.476541 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:33.476707 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:33.477152 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:33.477155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:33.477157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:33.477947 (+ 790us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:33.478192 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:33.478775 (+ 583us) server_negotiation.cc:299] Negotiation successful
0807 19:42:33.479443 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":352,"threads_started":1}
I20260807 19:42:33.481562 1980 raft_consensus.cc:504] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d489b62c4b9849c0840567f2f7df2e53" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43303 } }
I20260807 19:42:33.482080 1980 leader_election.cc:302] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d489b62c4b9849c0840567f2f7df2e53; no voters:
I20260807 19:42:33.482752 1980 leader_election.cc:288] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:33.482899 1985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:33.484182 1985 raft_consensus.cc:686] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [term 1 LEADER]: Becoming Leader. State: Replica: d489b62c4b9849c0840567f2f7df2e53, State: Running, Role: LEADER
I20260807 19:42:33.484671 1985 consensus_queue.cc:237] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d489b62c4b9849c0840567f2f7df2e53" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43303 } }
I20260807 19:42:33.484944 1980 sys_catalog.cc:564] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:33.489073 1986 sys_catalog.cc:455] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d489b62c4b9849c0840567f2f7df2e53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d489b62c4b9849c0840567f2f7df2e53" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43303 } } }
I20260807 19:42:33.489562 1986 sys_catalog.cc:458] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:33.489516 1987 sys_catalog.cc:455] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d489b62c4b9849c0840567f2f7df2e53. Latest consensus state: current_term: 1 leader_uuid: "d489b62c4b9849c0840567f2f7df2e53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d489b62c4b9849c0840567f2f7df2e53" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43303 } } }
I20260807 19:42:33.489786 1987 sys_catalog.cc:458] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:33.491797 1994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:33.496948 1994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:33.505573 1994 catalog_manager.cc:1347] Generated new cluster ID: bd655d35925b4e25a96f6879685d0f2b
I20260807 19:42:33.505703 1994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:33.515321 1994 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:33.516606 1994 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:33.538039 1994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53: Generated new TSK 0
I20260807 19:42:33.538925 1994 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
Entry for principal kudu/127.14.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab.
I20260807 19:42:33.612528 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43303
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:33.955353 2008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:33.955621 2008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:33.955814 2008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:33.963032 2008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:33.963155 2008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:33.963394 2008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:42:33.973059 2008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.65.keytab
--principal=kudu/127.14.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:33.974287 2008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:33.975653 2008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:33.989022 2013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:33.989097 2016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:33.989058 2014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:33.989858 2008 server_base.cc:1034] running on GCE node
I20260807 19:42:33.990620 2008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:33.991580 2008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:33.992782 2008 hybrid_clock.cc:648] HybridClock initialized: now 1786131753992746 us; error 31 us; skew 500 ppm
I20260807 19:42:33.993156 2008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:33 dist-test-slave-07c2 krb5kdc[1899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131753, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:33.996431 2008 init.cc:377] Logged in from keytab as kudu/127.14.16.65@KRBTEST.COM (short username kudu)
I20260807 19:42:33.997799 2008 webserver.cc:460] Webserver started at http://127.14.16.65:40827/ using document root <none> and password file <none>
I20260807 19:42:33.998423 2008 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:33.998521 2008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:33.998796 2008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:34.001652 2008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "d76751eae91147a5abbb0fd341cff6a7"
format_stamp: "Formatted at 2026-08-07 19:42:33 on dist-test-slave-07c2"
server_key: "9c4e8717ff0c91e6a6025f8fdf43e7f9"
server_key_iv: "ec8cd6b7d2c159e8b2f75421ce1230ca"
server_key_version: "encryptionkey@0"
I20260807 19:42:34.002373 2008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "d76751eae91147a5abbb0fd341cff6a7"
format_stamp: "Formatted at 2026-08-07 19:42:33 on dist-test-slave-07c2"
server_key: "9c4e8717ff0c91e6a6025f8fdf43e7f9"
server_key_iv: "ec8cd6b7d2c159e8b2f75421ce1230ca"
server_key_version: "encryptionkey@0"
I20260807 19:42:34.007520 2008 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:42:34.011020 2023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:34.012409 2008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:34.012584 2008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "d76751eae91147a5abbb0fd341cff6a7"
format_stamp: "Formatted at 2026-08-07 19:42:33 on dist-test-slave-07c2"
server_key: "9c4e8717ff0c91e6a6025f8fdf43e7f9"
server_key_iv: "ec8cd6b7d2c159e8b2f75421ce1230ca"
server_key_version: "encryptionkey@0"
I20260807 19:42:34.012821 2008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:34.025962 2008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:34.029698 2008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:34.029987 2008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:34.030905 2008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:34.032347 2008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:34.032451 2008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:34.032600 2008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:34.032693 2008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:34.065925 2008 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:35899
I20260807 19:42:34.065974 2137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:35899 every 8 connection(s)
I20260807 19:42:34.067214 2008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:34.073926 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 2008
I20260807 19:42:34.074108 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:34.074664 14401 external_mini_cluster.cc:1428] Setting key b664ad3dd526bbcc8c2875a5f569cdd3
Aug 07 19:42:34 dist-test-slave-07c2 krb5kdc[1899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131753, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.65@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:42:34.089134 2140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:34.069815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:55501 (local address 127.14.16.126:43303)
0807 19:42:34.070303 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:34.070308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:34.071070 (+ 762us) server_negotiation.cc:407] Connection header received
0807 19:42:34.072928 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:34.072934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:34.073039 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:34.073158 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:34.075147 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.076071 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.078348 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.078768 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.078869 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:34.083015 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:34.083020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:34.083023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:34.083056 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:34.085853 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:34.086677 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:34.086684 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:34.086692 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:34.086817 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:34.087249 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:34.087256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:34.087259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:34.087590 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:34.087771 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:34.088483 (+ 712us) server_negotiation.cc:299] Negotiation successful
0807 19:42:34.088740 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":254,"threads_started":1}
I20260807 19:42:34.090420 2141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:34.070193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:43303 (local address 127.14.16.65:55501)
0807 19:42:34.070903 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:34.070950 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:34.072656 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:34.073398 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:34.073406 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:34.074172 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:34.074963 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:34.074988 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.076258 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.076262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:34.078184 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:34.078202 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.079004 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.079014 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:34.079384 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:34.080135 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:34.080170 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:34.082795 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:34.086038 (+ 3243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:34.086045 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:34.086058 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:34.086547 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:34.086929 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:34.086935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:34.086940 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:34.087126 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:34.087729 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:34.087739 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:34.088111 (+ 372us) client_negotiation.cc:769] Sending connection context
0807 19:42:34.088390 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 19:42:34.088969 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":281,"threads_started":1}
I20260807 19:42:34.092147 2138 heartbeater.cc:347] Connected to a master server at 127.14.16.126:43303
I20260807 19:42:34.092496 2138 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:34.093364 2138 heartbeater.cc:511] Master 127.14.16.126:43303 requested a full tablet report, sending...
I20260807 19:42:34.095837 1944 ts_manager.cc:194] Registered new tserver with Master: d76751eae91147a5abbb0fd341cff6a7 (127.14.16.65:35899)
I20260807 19:42:34.097532 1944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.65@KRBTEST.COM'} at 127.14.16.65:55501
WARNING: no policy specified for kudu/127.14.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
Entry for principal kudu/127.14.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab.
I20260807 19:42:34.145911 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43303
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:34.487314 2144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:34.487584 2144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:34.487746 2144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:34.495179 2144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:34.495304 2144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:34.495481 2144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:42:34.505041 2144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.66.keytab
--principal=kudu/127.14.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:34.506314 2144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:34.507622 2144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:34.520628 2149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:34.521064 2152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:34.521124 2150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:34.521994 2144 server_base.cc:1034] running on GCE node
I20260807 19:42:34.522580 2144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:34.523538 2144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:34.524752 2144 hybrid_clock.cc:648] HybridClock initialized: now 1786131754524707 us; error 31 us; skew 500 ppm
I20260807 19:42:34.525221 2144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:34 dist-test-slave-07c2 krb5kdc[1899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131754, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:34.529135 2144 init.cc:377] Logged in from keytab as kudu/127.14.16.66@KRBTEST.COM (short username kudu)
I20260807 19:42:34.530606 2144 webserver.cc:460] Webserver started at http://127.14.16.66:39639/ using document root <none> and password file <none>
I20260807 19:42:34.531250 2144 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:34.531424 2144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:34.531694 2144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:34.534580 2144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "019003c60f074e859f9fd69d937d7448"
format_stamp: "Formatted at 2026-08-07 19:42:34 on dist-test-slave-07c2"
server_key: "7a9222e65d3e844eca4eaae04afe4d04"
server_key_iv: "719f28c116a7f94f35739bc324eb4f6a"
server_key_version: "encryptionkey@0"
I20260807 19:42:34.535418 2144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "019003c60f074e859f9fd69d937d7448"
format_stamp: "Formatted at 2026-08-07 19:42:34 on dist-test-slave-07c2"
server_key: "7a9222e65d3e844eca4eaae04afe4d04"
server_key_iv: "719f28c116a7f94f35739bc324eb4f6a"
server_key_version: "encryptionkey@0"
I20260807 19:42:34.540575 2144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 19:42:34.544027 2159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:34.545485 2144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:34.545629 2144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "019003c60f074e859f9fd69d937d7448"
format_stamp: "Formatted at 2026-08-07 19:42:34 on dist-test-slave-07c2"
server_key: "7a9222e65d3e844eca4eaae04afe4d04"
server_key_iv: "719f28c116a7f94f35739bc324eb4f6a"
server_key_version: "encryptionkey@0"
I20260807 19:42:34.545795 2144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:34.569135 2144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:34.573400 2144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:34.573658 2144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:34.574595 2144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:34.576005 2144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:34.576085 2144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:34.576170 2144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:34.576215 2144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:34.610054 2144 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:43925
I20260807 19:42:34.610134 2271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:43925 every 8 connection(s)
I20260807 19:42:34.611439 2144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:34.616847 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 2144
I20260807 19:42:34.617048 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:34.617607 14401 external_mini_cluster.cc:1428] Setting key 50b808cc7714ae64e06480ca60d4672e
Aug 07 19:42:34 dist-test-slave-07c2 krb5kdc[1899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131754, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.66@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:42:34.635849 2275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:34.614545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:43303 (local address 127.14.16.66:52601)
0807 19:42:34.615269 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:34.615376 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:34.617370 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:34.618190 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:34.618204 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:34.618940 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:34.619894 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:34.619921 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.620968 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.620974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:34.622676 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:34.622702 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.623909 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.623915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:34.624162 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:34.624885 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:34.624908 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:34.627257 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:34.631378 (+ 4121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:34.631385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:34.631399 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:34.631859 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:34.632279 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:34.632283 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:34.632285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:34.632468 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:34.633202 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:34.633213 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:34.633747 (+ 534us) client_negotiation.cc:769] Sending connection context
0807 19:42:34.634078 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 19:42:34.634564 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":290,"threads_started":1}
I20260807 19:42:34.636148 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:34.614024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:52601 (local address 127.14.16.126:43303)
0807 19:42:34.614477 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:34.614481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:34.615503 (+ 1022us) server_negotiation.cc:407] Connection header received
0807 19:42:34.617651 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:34.617657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:34.617749 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:34.617885 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:34.620080 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.620804 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.622986 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.623662 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.623805 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:34.627534 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:34.627544 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:34.627551 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:34.627599 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:34.631180 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:34.631982 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:34.631990 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:34.631996 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:34.632157 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:34.632589 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:34.632596 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:34.632602 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:34.632999 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:34.633182 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:34.635524 (+ 2342us) server_negotiation.cc:299] Negotiation successful
0807 19:42:34.635768 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":229,"threads_started":1}
I20260807 19:42:34.637636 2272 heartbeater.cc:347] Connected to a master server at 127.14.16.126:43303
I20260807 19:42:34.637981 2272 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:34.638677 2272 heartbeater.cc:511] Master 127.14.16.126:43303 requested a full tablet report, sending...
I20260807 19:42:34.640134 1944 ts_manager.cc:194] Registered new tserver with Master: 019003c60f074e859f9fd69d937d7448 (127.14.16.66:43925)
I20260807 19:42:34.640936 1944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.66@KRBTEST.COM'} at 127.14.16.66:52601
WARNING: no policy specified for kudu/127.14.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
Entry for principal kudu/127.14.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab.
I20260807 19:42:34.690868 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43303
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:35.036478 2278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:35.036803 2278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:35.036902 2278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:35.044463 2278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:35.044588 2278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:35.044713 2278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:42:35.055050 2278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.67.keytab
--principal=kudu/127.14.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:43303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:35.056406 2278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:35.057674 2278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:35.070493 2283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:35.070571 2284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:35.070565 2286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:35.072263 2278 server_base.cc:1034] running on GCE node
I20260807 19:42:35.072800 2278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:35.073642 2278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:35.074857 2278 hybrid_clock.cc:648] HybridClock initialized: now 1786131755074816 us; error 32 us; skew 500 ppm
I20260807 19:42:35.075173 2278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:35 dist-test-slave-07c2 krb5kdc[1899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131755, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:35.078295 2278 init.cc:377] Logged in from keytab as kudu/127.14.16.67@KRBTEST.COM (short username kudu)
I20260807 19:42:35.079905 2278 webserver.cc:460] Webserver started at http://127.14.16.67:36727/ using document root <none> and password file <none>
I20260807 19:42:35.080591 2278 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:35.080732 2278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:35.081064 2278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:35.084169 2278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "4c95aa4b20af4e25814fdf87bb41fadb"
format_stamp: "Formatted at 2026-08-07 19:42:35 on dist-test-slave-07c2"
server_key: "ba165013c161b7b0bc2428ef1cc998c1"
server_key_iv: "fb1ecbc338d0201297c4714506d19a61"
server_key_version: "encryptionkey@0"
I20260807 19:42:35.084944 2278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "4c95aa4b20af4e25814fdf87bb41fadb"
format_stamp: "Formatted at 2026-08-07 19:42:35 on dist-test-slave-07c2"
server_key: "ba165013c161b7b0bc2428ef1cc998c1"
server_key_iv: "fb1ecbc338d0201297c4714506d19a61"
server_key_version: "encryptionkey@0"
I20260807 19:42:35.090585 2278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:42:35.094091 2293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:35.095484 2278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:35.095656 2278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "4c95aa4b20af4e25814fdf87bb41fadb"
format_stamp: "Formatted at 2026-08-07 19:42:35 on dist-test-slave-07c2"
server_key: "ba165013c161b7b0bc2428ef1cc998c1"
server_key_iv: "fb1ecbc338d0201297c4714506d19a61"
server_key_version: "encryptionkey@0"
I20260807 19:42:35.095865 2278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:35.101248 2138 heartbeater.cc:503] Master 127.14.16.126:43303 was elected leader, sending a full tablet report...
I20260807 19:42:35.123149 2278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:35.127229 2278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:35.127503 2278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:35.128451 2278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:35.129736 2278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:35.129809 2278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:35.129892 2278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:35.129944 2278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:35.163739 2278 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:37293
I20260807 19:42:35.163811 2405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:37293 every 8 connection(s)
I20260807 19:42:35.165035 2278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:35.174496 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 2278
I20260807 19:42:35.174726 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:35.175210 14401 external_mini_cluster.cc:1428] Setting key 903c7a39eb4b9d9a960e02c536e3b2eb
Aug 07 19:42:35 dist-test-slave-07c2 krb5kdc[1899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131755, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.67@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:42:35.184747 2408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.167519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:51407 (local address 127.14.16.126:43303)
0807 19:42:35.167939 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.167944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.168943 (+ 999us) server_negotiation.cc:407] Connection header received
0807 19:42:35.170779 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.170784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.170873 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.170995 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:35.172490 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.173277 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.174839 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.175260 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.175418 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.179188 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:35.179193 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:35.179196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:35.179237 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:35.182050 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.182724 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.182728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.182730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.182851 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.183224 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.183228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.183230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.183538 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:35.183651 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.184242 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.184401 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":210,"threads_started":1}
I20260807 19:42:35.185765 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.168062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:43303 (local address 127.14.16.67:51407)
0807 19:42:35.168769 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:35.168808 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:35.170517 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:35.171232 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:35.171239 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:35.171788 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:35.172324 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.172341 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.173423 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.173429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.174695 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.174716 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.175534 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.175542 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.175820 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.176539 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:35.176578 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:35.179001 (+ 2423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:35.182191 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:35.182198 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:35.182210 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:35.182620 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:35.182944 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:35.182947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:35.182949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:35.183133 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:35.183642 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:35.183647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:35.183951 (+ 304us) client_negotiation.cc:769] Sending connection context
0807 19:42:35.184168 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 19:42:35.184580 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":273,"threads_started":1}
I20260807 19:42:35.187084 2406 heartbeater.cc:347] Connected to a master server at 127.14.16.126:43303
I20260807 19:42:35.187445 2406 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:35.188145 2406 heartbeater.cc:511] Master 127.14.16.126:43303 requested a full tablet report, sending...
I20260807 19:42:35.189847 1943 ts_manager.cc:194] Registered new tserver with Master: 4c95aa4b20af4e25814fdf87bb41fadb (127.14.16.67:37293)
I20260807 19:42:35.190758 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:42:35.191009 1943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.16.67@KRBTEST.COM'} at 127.14.16.67:51407
I20260807 19:42:35.208227 2408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.195383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39856 (local address 127.14.16.126:43303)
0807 19:42:35.195589 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.195594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.196060 (+ 466us) server_negotiation.cc:407] Connection header received
0807 19:42:35.196149 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.196152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.196213 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.196298 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:35.197635 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.198361 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.199895 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.200537 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.200662 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.203496 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:35.203500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:35.203503 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:35.203534 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:35.205863 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.206476 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.206480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.206482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.206604 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.206917 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.206920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.206921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.207220 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:35.207584 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.207826 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.207998 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:42:35.212138 1943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:35.219060 1943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:35.240586 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.230726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:37293 (local address 127.0.0.1:39962)
0807 19:42:35.231846 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:35.231889 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:35.232067 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:35.233588 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:35.233594 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:35.233635 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:35.234061 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.234075 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.235981 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.235988 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.238149 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.238161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.239486 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.239494 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.239717 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.239792 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 19:42:35.239903 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 19:42:35.240110 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":3,"thread_start_us":795,"threads_started":1}
I20260807 19:42:35.241497 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.231225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39962 (local address 127.14.16.67:37293)
0807 19:42:35.232862 (+ 1637us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.232872 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.232906 (+ 34us) server_negotiation.cc:407] Connection header received
0807 19:42:35.233011 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.233018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.233298 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.234181 (+ 883us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:35.234422 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.235858 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.238304 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.239249 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.239434 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.239700 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.240989 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.241155 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1430,"thread_start_us":1228,"threads_started":1}
I20260807 19:42:35.242303 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.230940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:43925 (local address 127.0.0.1:49356)
0807 19:42:35.233158 (+ 2218us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:35.233179 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:35.233306 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:35.234988 (+ 1682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:35.234994 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:35.235019 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:35.235502 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.235519 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.237339 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.237346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.239796 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.239808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.241512 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.241519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.241722 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.241753 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:42:35.241849 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 19:42:35.241981 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2077,"thread_start_us":4539,"threads_started":1}
I20260807 19:42:35.242519 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.231361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49356 (local address 127.14.16.66:43925)
0807 19:42:35.234328 (+ 2967us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.234340 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.234374 (+ 34us) server_negotiation.cc:407] Connection header received
0807 19:42:35.234486 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.234496 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.234744 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.234984 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:35.235728 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.237198 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.239983 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.241288 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.241451 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.241748 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.241916 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.242081 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2811,"thread_start_us":2713,"threads_started":1}
I20260807 19:42:35.245121 2421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.232063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:35899 (local address 127.0.0.1:60542)
0807 19:42:35.235756 (+ 3693us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:35.235769 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:35.235900 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:35.236734 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:35.236739 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:35.236773 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:35.237336 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.237351 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.239342 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.239350 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.241913 (+ 2563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.241928 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.244353 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.244359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.244530 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.244560 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:42:35.244648 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 19:42:35.244768 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3562,"mutex_wait_us":51,"thread_start_us":792,"threads_started":1}
I20260807 19:42:35.245395 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.231764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.14.16.65:35899)
0807 19:42:35.233924 (+ 2160us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.233935 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.235866 (+ 1931us) server_negotiation.cc:407] Connection header received
0807 19:42:35.236081 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.236090 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.236309 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.236482 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:35.237568 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.239163 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.242092 (+ 2929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.243849 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.243977 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.244191 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.244924 (+ 733us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.245084 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1956,"thread_start_us":1795,"threads_started":1}
I20260807 19:42:35.245524 2341 tablet_service.cc:1467] Processing CreateTablet for tablet 6ae2d5ade458488da8ed00b72bbcde98 (DEFAULT_TABLE table=test-table [id=86c43b5be26c4c5985735f1f1e272590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:35.245726 2207 tablet_service.cc:1467] Processing CreateTablet for tablet 6ae2d5ade458488da8ed00b72bbcde98 (DEFAULT_TABLE table=test-table [id=86c43b5be26c4c5985735f1f1e272590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:35.246919 2207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ae2d5ade458488da8ed00b72bbcde98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:35.247002 2341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ae2d5ade458488da8ed00b72bbcde98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:35.248811 2072 tablet_service.cc:1467] Processing CreateTablet for tablet 6ae2d5ade458488da8ed00b72bbcde98 (DEFAULT_TABLE table=test-table [id=86c43b5be26c4c5985735f1f1e272590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:35.250125 2072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ae2d5ade458488da8ed00b72bbcde98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:35.258505 2426 tablet_bootstrap.cc:492] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448: Bootstrap starting.
I20260807 19:42:35.261125 2427 tablet_bootstrap.cc:492] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb: Bootstrap starting.
I20260807 19:42:35.262195 2426 tablet_bootstrap.cc:654] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:35.263602 2426 log.cc:824] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:35.264168 2428 tablet_bootstrap.cc:492] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7: Bootstrap starting.
I20260807 19:42:35.264708 2427 tablet_bootstrap.cc:654] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:35.265829 2427 log.cc:824] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:35.267647 2428 tablet_bootstrap.cc:654] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:35.267845 2426 tablet_bootstrap.cc:492] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448: No bootstrap required, opened a new log
I20260807 19:42:35.268137 2426 ts_tablet_manager.cc:1397] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 19:42:35.268479 2427 tablet_bootstrap.cc:492] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb: No bootstrap required, opened a new log
I20260807 19:42:35.268747 2427 ts_tablet_manager.cc:1397] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:42:35.268882 2428 log.cc:824] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:35.272667 2428 tablet_bootstrap.cc:492] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7: No bootstrap required, opened a new log
I20260807 19:42:35.272945 2428 ts_tablet_manager.cc:1397] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:42:35.276671 2427 raft_consensus.cc:348] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37293 } } peers { permanent_uuid: "019003c60f074e859f9fd69d937d7448" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 43925 } } peers { permanent_uuid: "d76751eae91147a5abbb0fd341cff6a7" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 35899 } }
I20260807 19:42:35.277058 2427 raft_consensus.cc:374] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:35.277165 2427 raft_consensus.cc:729] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c95aa4b20af4e25814fdf87bb41fadb, State: Initialized, Role: FOLLOWER
I20260807 19:42:35.277796 2427 consensus_queue.cc:260] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37293 } } peers { permanent_uuid: "019003c60f074e859f9fd69d937d7448" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 43925 } } peers { permanent_uuid: "d76751eae91147a5abbb0fd341cff6a7" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 35899 } }
I20260807 19:42:35.278589 2426 raft_consensus.cc:348] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37293 } } peers { permanent_uuid: "019003c60f074e859f9fd69d937d7448" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 43925 } } peers { permanent_uuid: "d76751eae91147a5abbb0fd341cff6a7" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 35899 } }
I20260807 19:42:35.278976 2406 heartbeater.cc:503] Master 127.14.16.126:43303 was elected leader, sending a full tablet report...
I20260807 19:42:35.279004 2426 raft_consensus.cc:374] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:35.279088 2426 raft_consensus.cc:729] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019003c60f074e859f9fd69d937d7448, State: Initialized, Role: FOLLOWER
I20260807 19:42:35.279680 2427 ts_tablet_manager.cc:1428] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260807 19:42:35.279757 2426 consensus_queue.cc:260] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37293 } } peers { permanent_uuid: "019003c60f074e859f9fd69d937d7448" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 43925 } } peers { permanent_uuid: "d76751eae91147a5abbb0fd341cff6a7" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 35899 } }
I20260807 19:42:35.280895 2272 heartbeater.cc:503] Master 127.14.16.126:43303 was elected leader, sending a full tablet report...
I20260807 19:42:35.281505 2426 ts_tablet_manager.cc:1428] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260807 19:42:35.285045 2428 raft_consensus.cc:348] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37293 } } peers { permanent_uuid: "019003c60f074e859f9fd69d937d7448" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 43925 } } peers { permanent_uuid: "d76751eae91147a5abbb0fd341cff6a7" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 35899 } }
I20260807 19:42:35.285529 2428 raft_consensus.cc:374] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:35.285621 2428 raft_consensus.cc:729] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d76751eae91147a5abbb0fd341cff6a7, State: Initialized, Role: FOLLOWER
I20260807 19:42:35.286237 2428 consensus_queue.cc:260] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37293 } } peers { permanent_uuid: "019003c60f074e859f9fd69d937d7448" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 43925 } } peers { permanent_uuid: "d76751eae91147a5abbb0fd341cff6a7" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 35899 } }
I20260807 19:42:35.292917 2428 ts_tablet_manager.cc:1428] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7: Time spent starting tablet: real 0.020s user 0.019s sys 0.000s
W20260807 19:42:35.319576 2139 tablet.cc:2369] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:35.332446 2432 raft_consensus.cc:482] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:35.332726 2432 raft_consensus.cc:504] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37293 } } peers { permanent_uuid: "019003c60f074e859f9fd69d937d7448" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 43925 } } peers { permanent_uuid: "d76751eae91147a5abbb0fd341cff6a7" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 35899 } }
I20260807 19:42:35.334291 2432 leader_election.cc:288] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019003c60f074e859f9fd69d937d7448 (127.14.16.66:43925), d76751eae91147a5abbb0fd341cff6a7 (127.14.16.65:35899)
I20260807 19:42:35.341020 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.334747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:32879 (local address 127.14.16.66:43925)
0807 19:42:35.334940 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.334946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.334969 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:42:35.335254 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.335261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.335385 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.335493 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:35.335964 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.337015 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.338931 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.340184 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.340301 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.340353 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.340727 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.340826 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:42:35.341117 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.334532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:43925 (local address 127.14.16.67:32879)
0807 19:42:35.334863 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:35.334887 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:35.335023 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:35.335535 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:35.335538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:35.335561 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:35.335740 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.335747 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.337161 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.337167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.338770 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.338782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.340335 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.340342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.340538 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.340575 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:42:35.340676 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 19:42:35.340829 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":2}
I20260807 19:42:35.342226 2435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.334645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:35899 (local address 127.14.16.67:36155)
0807 19:42:35.335736 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:35.335753 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:35.335935 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:35.336260 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:35.336266 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:35.336293 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:35.336675 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.336687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.337985 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.337988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.340018 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.340033 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.341626 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.341633 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.341780 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.341803 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:42:35.341882 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 19:42:35.341959 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"mutex_wait_us":39,"thread_start_us":590,"threads_started":1}
I20260807 19:42:35.342345 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.334744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:36155 (local address 127.14.16.65:35899)
0807 19:42:35.334940 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.334946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.335867 (+ 921us) server_negotiation.cc:407] Connection header received
0807 19:42:35.336080 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.336085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.336144 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.336208 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:35.336827 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.337827 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.340192 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.341445 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.341558 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.341605 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.341946 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.342078 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:42:35.342192 2227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ae2d5ade458488da8ed00b72bbcde98" candidate_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019003c60f074e859f9fd69d937d7448" is_pre_election: true
I20260807 19:42:35.342782 2227 raft_consensus.cc:2454] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c95aa4b20af4e25814fdf87bb41fadb in term 0.
I20260807 19:42:35.343217 2093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ae2d5ade458488da8ed00b72bbcde98" candidate_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d76751eae91147a5abbb0fd341cff6a7" is_pre_election: true
I20260807 19:42:35.343504 2297 leader_election.cc:302] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019003c60f074e859f9fd69d937d7448, 4c95aa4b20af4e25814fdf87bb41fadb; no voters:
I20260807 19:42:35.343775 2093 raft_consensus.cc:2454] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c95aa4b20af4e25814fdf87bb41fadb in term 0.
I20260807 19:42:35.343940 2432 raft_consensus.cc:2781] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:35.344089 2432 raft_consensus.cc:482] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:35.344197 2432 raft_consensus.cc:3037] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:35.346442 2432 raft_consensus.cc:504] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37293 } } peers { permanent_uuid: "019003c60f074e859f9fd69d937d7448" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 43925 } } peers { permanent_uuid: "d76751eae91147a5abbb0fd341cff6a7" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 35899 } }
I20260807 19:42:35.347071 2432 leader_election.cc:288] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [CANDIDATE]: Term 1 election: Requested vote from peers 019003c60f074e859f9fd69d937d7448 (127.14.16.66:43925), d76751eae91147a5abbb0fd341cff6a7 (127.14.16.65:35899)
I20260807 19:42:35.347452 2227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ae2d5ade458488da8ed00b72bbcde98" candidate_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019003c60f074e859f9fd69d937d7448"
I20260807 19:42:35.347553 2093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ae2d5ade458488da8ed00b72bbcde98" candidate_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d76751eae91147a5abbb0fd341cff6a7"
I20260807 19:42:35.347671 2227 raft_consensus.cc:3037] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:35.347751 2093 raft_consensus.cc:3037] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:35.350978 2093 raft_consensus.cc:2454] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c95aa4b20af4e25814fdf87bb41fadb in term 1.
I20260807 19:42:35.350978 2227 raft_consensus.cc:2454] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c95aa4b20af4e25814fdf87bb41fadb in term 1.
I20260807 19:42:35.351394 2294 leader_election.cc:302] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c95aa4b20af4e25814fdf87bb41fadb, d76751eae91147a5abbb0fd341cff6a7; no voters:
I20260807 19:42:35.351707 2432 raft_consensus.cc:2781] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:35.352185 2432 raft_consensus.cc:686] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 1 LEADER]: Becoming Leader. State: Replica: 4c95aa4b20af4e25814fdf87bb41fadb, State: Running, Role: LEADER
I20260807 19:42:35.352619 2432 consensus_queue.cc:237] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37293 } } peers { permanent_uuid: "019003c60f074e859f9fd69d937d7448" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 43925 } } peers { permanent_uuid: "d76751eae91147a5abbb0fd341cff6a7" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 35899 } }
I20260807 19:42:35.357347 1944 catalog_manager.cc:5495] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c95aa4b20af4e25814fdf87bb41fadb (127.14.16.67). New cstate: current_term: 1 leader_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c95aa4b20af4e25814fdf87bb41fadb" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 37293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019003c60f074e859f9fd69d937d7448" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 43925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d76751eae91147a5abbb0fd341cff6a7" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 35899 } health_report { overall_health: UNKNOWN } } }
W20260807 19:42:35.363458 2273 tablet.cc:2369] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:35.398188 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.392933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39972 (local address 127.14.16.67:37293)
0807 19:42:35.393175 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.393183 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.393246 (+ 63us) server_negotiation.cc:407] Connection header received
0807 19:42:35.393453 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.393457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.393536 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.393660 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:35.394085 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.394831 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.396309 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.396704 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.396795 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.397240 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:35.397728 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:35.397834 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.397930 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.397993 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:42:35.405843 2093 raft_consensus.cc:1261] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7 [term 1 FOLLOWER]: Refusing update from remote peer 4c95aa4b20af4e25814fdf87bb41fadb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:35.405860 2227 raft_consensus.cc:1261] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448 [term 1 FOLLOWER]: Refusing update from remote peer 4c95aa4b20af4e25814fdf87bb41fadb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:35.406777 2436 consensus_queue.cc:1035] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d76751eae91147a5abbb0fd341cff6a7" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 35899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:35.407143 2432 consensus_queue.cc:1035] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [LEADER]: Connected to new peer: Peer: permanent_uuid: "019003c60f074e859f9fd69d937d7448" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 43925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:42:35.416702 2407 tablet.cc:2369] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:35.420248 2439 mvcc.cc:205] Tried to move back new op lower bound from 7315995670133252096 to 7315995669928415232. Current Snapshot: MvccSnapshot[applied={T|T < 7315995670133252096}]
I20260807 19:42:35.436581 1942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:35.436915 1942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:35.440815 1942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d489b62c4b9849c0840567f2f7df2e53: Sending DeleteTablet for 3 replicas of tablet 6ae2d5ade458488da8ed00b72bbcde98
I20260807 19:42:35.442049 2341 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ae2d5ade458488da8ed00b72bbcde98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:35 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:39962
I20260807 19:42:35.442042 2207 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ae2d5ade458488da8ed00b72bbcde98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:35 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:49356
I20260807 19:42:35.442149 2072 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ae2d5ade458488da8ed00b72bbcde98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:35 UTC) from {username='kudu', principal='kudu/127.14.16.126@KRBTEST.COM'} at 127.0.0.1:60542
I20260807 19:42:35.451331 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 2008
I20260807 19:42:35.452289 2451 tablet_replica.cc:330] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7: stopping tablet replica
I20260807 19:42:35.452772 2451 raft_consensus.cc:2229] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:35.453329 2451 raft_consensus.cc:2258] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:35.454394 2449 tablet_replica.cc:330] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448: stopping tablet replica
I20260807 19:42:35.454845 2449 raft_consensus.cc:2229] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:35.455344 2449 raft_consensus.cc:2258] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:35.456172 2450 tablet_replica.cc:330] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb: stopping tablet replica
I20260807 19:42:35.456557 2450 raft_consensus.cc:2229] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:35.457321 2450 raft_consensus.cc:2258] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:35.459465 2449 ts_tablet_manager.cc:1905] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:35.461087 2451 ts_tablet_manager.cc:1905] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:35.466547 2449 ts_tablet_manager.cc:1918] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:35.466722 2449 log.cc:1196] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/6ae2d5ade458488da8ed00b72bbcde98
I20260807 19:42:35.469501 2450 ts_tablet_manager.cc:1905] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:35.476339 2451 ts_tablet_manager.cc:1918] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:35.476541 2451 log.cc:1196] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/6ae2d5ade458488da8ed00b72bbcde98
I20260807 19:42:35.476959 2450 ts_tablet_manager.cc:1918] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:35.477147 2450 log.cc:1196] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/6ae2d5ade458488da8ed00b72bbcde98
I20260807 19:42:35.477154 2451 ts_tablet_manager.cc:1939] T 6ae2d5ade458488da8ed00b72bbcde98 P d76751eae91147a5abbb0fd341cff6a7: Deleting consensus metadata
I20260807 19:42:35.477703 2450 ts_tablet_manager.cc:1939] T 6ae2d5ade458488da8ed00b72bbcde98 P 4c95aa4b20af4e25814fdf87bb41fadb: Deleting consensus metadata
I20260807 19:42:35.478248 2449 ts_tablet_manager.cc:1939] T 6ae2d5ade458488da8ed00b72bbcde98 P 019003c60f074e859f9fd69d937d7448: Deleting consensus metadata
I20260807 19:42:35.479249 1928 catalog_manager.cc:4841] TS d76751eae91147a5abbb0fd341cff6a7 (127.14.16.65:35899): tablet 6ae2d5ade458488da8ed00b72bbcde98 (table test-table [id=86c43b5be26c4c5985735f1f1e272590]) successfully deleted
I20260807 19:42:35.480167 1931 catalog_manager.cc:4841] TS 019003c60f074e859f9fd69d937d7448 (127.14.16.66:43925): tablet 6ae2d5ade458488da8ed00b72bbcde98 (table test-table [id=86c43b5be26c4c5985735f1f1e272590]) successfully deleted
I20260807 19:42:35.481346 1929 catalog_manager.cc:4841] TS 4c95aa4b20af4e25814fdf87bb41fadb (127.14.16.67:37293): tablet 6ae2d5ade458488da8ed00b72bbcde98 (table test-table [id=86c43b5be26c4c5985735f1f1e272590]) successfully deleted
Aug 07 19:42:35 dist-test-slave-07c2 krb5kdc[1899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.65@KRBTEST.COM
I20260807 19:42:35.491925 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.453128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60550 (local address 127.14.16.65:35899)
0807 19:42:35.453368 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.453378 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.470505 (+ 17127us) server_negotiation.cc:407] Connection header received
0807 19:42:35.470800 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.470807 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.470889 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.470968 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:35.472861 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.474032 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.475985 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.476662 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.476817 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.484092 (+ 7275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:35.484110 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:35.484129 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:35.484190 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:35.488107 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.489122 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.489132 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.489136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.489313 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.489799 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.489804 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.489808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.490335 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:35.490626 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.491244 (+ 618us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.491513 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:42:35.492713 2133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:35.626288 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 2008
I20260807 19:42:35.651806 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 2144
Aug 07 19:42:35 dist-test-slave-07c2 krb5kdc[1899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.66@KRBTEST.COM
I20260807 19:42:35.669912 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.652732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49370 (local address 127.14.16.66:43925)
0807 19:42:35.652899 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.652903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.652921 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:42:35.653085 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.653093 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.653179 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.653260 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:35.655096 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.655951 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.657874 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.658279 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.658376 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.663707 (+ 5331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:35.663721 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:35.663737 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:35.663775 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:35.666871 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.667734 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.667742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.667744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.667871 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.668255 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.668261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.668264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.668671 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:35.668968 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.669316 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.669538 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 19:42:35.670840 2267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:35.762984 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 2144
I20260807 19:42:35.782647 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 2278
Aug 07 19:42:35 dist-test-slave-07c2 krb5kdc[1899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.67@KRBTEST.COM
I20260807 19:42:35.800470 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.783688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39982 (local address 127.14.16.67:37293)
0807 19:42:35.783896 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.783904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.783928 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:42:35.783995 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.784001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.784111 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.784236 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:35.785616 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.786905 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.788202 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.788954 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.789129 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.793862 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:35.793874 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:35.793890 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:35.793940 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:35.797459 (+ 3519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.798202 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.798210 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.798212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.798367 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.798778 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.798781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.798783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.799149 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:35.799508 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.799917 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.800080 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:42:35.801311 2401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:35.926098 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 2278
I20260807 19:42:35.948484 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 1912
I20260807 19:42:35.949656 1975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:36.061076 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 1912
2026-08-07T19:42:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4550 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T19:42:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:36Z Disabled control of system clock
I20260807 19:42:36.153314 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:35391
--webserver_interface=127.14.16.126
--webserver_port=0
--builtin_ntp_servers=127.14.16.84:45571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:35391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:42:36.507704 2461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:36.508016 2461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:36.508177 2461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:36.515431 2461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:36.515561 2461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:36.515640 2461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:36.515738 2461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:36.515817 2461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:36.524947 2461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:35391
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:35391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:36.526253 2461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:36.528172 2461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:36.540055 2467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:36.540134 2466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:36.540113 2470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:36.541010 2461 server_base.cc:1034] running on GCE node
I20260807 19:42:36.541788 2461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:36.543220 2461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:36.544473 2461 hybrid_clock.cc:648] HybridClock initialized: now 1786131756544428 us; error 38 us; skew 500 ppm
I20260807 19:42:36.544821 2461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:36.547163 2461 webserver.cc:460] Webserver started at http://127.14.16.126:36311/ using document root <none> and password file <none>
I20260807 19:42:36.547911 2461 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:36.548036 2461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:36.548355 2461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:36.551681 2461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "d18ba4309bd94748b938bb5139f1b4de"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-07c2"
server_key: "3fb3a008663a616388d3785df825b694"
server_key_iv: "6628c6a623c542d03773a0092204519a"
server_key_version: "encryptionkey@0"
I20260807 19:42:36.552728 2461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "d18ba4309bd94748b938bb5139f1b4de"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-07c2"
server_key: "3fb3a008663a616388d3785df825b694"
server_key_iv: "6628c6a623c542d03773a0092204519a"
server_key_version: "encryptionkey@0"
I20260807 19:42:36.558173 2461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:42:36.561750 2476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:36.563162 2461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 19:42:36.563414 2461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "d18ba4309bd94748b938bb5139f1b4de"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-07c2"
server_key: "3fb3a008663a616388d3785df825b694"
server_key_iv: "6628c6a623c542d03773a0092204519a"
server_key_version: "encryptionkey@0"
I20260807 19:42:36.563673 2461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:36.590627 2461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:36.591545 2461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:36.591871 2461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:36.611519 2461 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:35391
I20260807 19:42:36.611548 2529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:35391 every 8 connection(s)
I20260807 19:42:36.612932 2461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:36.616953 2530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:36.622757 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 2461
I20260807 19:42:36.623039 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:36.623595 14401 external_mini_cluster.cc:1428] Setting key 15998a224c104b49a2f95277d20f9cbe
I20260807 19:42:36.627624 2530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de: Bootstrap starting.
I20260807 19:42:36.630976 2530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:36.632264 2530 log.cc:824] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:36.635442 2530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de: No bootstrap required, opened a new log
I20260807 19:42:36.637661 2534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.625587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54362 (local address 127.14.16.126:35391)
0807 19:42:36.626558 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.626569 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.626609 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:42:36.628397 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.628419 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.628861 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.629200 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:36.629840 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.631187 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.633151 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.633973 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.634099 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.635396 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:36.635405 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:36.635417 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:36.635996 (+ 579us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:36.636070 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:36.636091 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:36.636190 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:36.636422 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 19:42:36.636732 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":341,"threads_started":1}
I20260807 19:42:36.643510 2530 raft_consensus.cc:348] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18ba4309bd94748b938bb5139f1b4de" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 35391 } }
I20260807 19:42:36.643942 2530 raft_consensus.cc:374] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:36.644052 2530 raft_consensus.cc:729] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d18ba4309bd94748b938bb5139f1b4de, State: Initialized, Role: FOLLOWER
I20260807 19:42:36.644584 2530 consensus_queue.cc:260] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18ba4309bd94748b938bb5139f1b4de" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 35391 } }
I20260807 19:42:36.644853 2530 raft_consensus.cc:388] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:36.644994 2530 raft_consensus.cc:482] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:36.645205 2530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:36.647567 2530 raft_consensus.cc:504] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18ba4309bd94748b938bb5139f1b4de" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 35391 } }
I20260807 19:42:36.648064 2530 leader_election.cc:302] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d18ba4309bd94748b938bb5139f1b4de; no voters:
I20260807 19:42:36.648656 2530 leader_election.cc:288] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:36.648818 2536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:36.650218 2536 raft_consensus.cc:686] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [term 1 LEADER]: Becoming Leader. State: Replica: d18ba4309bd94748b938bb5139f1b4de, State: Running, Role: LEADER
I20260807 19:42:36.650760 2530 sys_catalog.cc:564] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:36.650730 2536 consensus_queue.cc:237] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18ba4309bd94748b938bb5139f1b4de" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 35391 } }
I20260807 19:42:36.653913 2538 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [sys.catalog]: SysCatalogTable state changed. Reason: New leader d18ba4309bd94748b938bb5139f1b4de. Latest consensus state: current_term: 1 leader_uuid: "d18ba4309bd94748b938bb5139f1b4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18ba4309bd94748b938bb5139f1b4de" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 35391 } } }
I20260807 19:42:36.653959 2537 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d18ba4309bd94748b938bb5139f1b4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18ba4309bd94748b938bb5139f1b4de" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 35391 } } }
I20260807 19:42:36.654458 2538 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:36.654464 2537 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:36.658347 2546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:36.662664 2546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:36.670768 2546 catalog_manager.cc:1347] Generated new cluster ID: 9b0e732520e04536af60a841a1e1bb65
I20260807 19:42:36.670878 2546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:36.678915 2546 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:36.680553 2546 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:36.691118 2546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de: Generated new TSK 0
I20260807 19:42:36.692027 2546 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:36.703030 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.65:0
--local_ip_for_outbound_sockets=127.14.16.65
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:35391
--builtin_ntp_servers=127.14.16.84:45571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:42:37.045414 2555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:37.045799 2555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:37.045958 2555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:37.053373 2555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:37.053471 2555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:37.053627 2555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.65
I20260807 19:42:37.062686 2555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.16.65
--webserver_port=0
--tserver_master_addrs=127.14.16.126:35391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.65
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:37.064033 2555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:37.066108 2555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:37.079770 2560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:37.079862 2561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:37.079830 2563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:37.080618 2555 server_base.cc:1034] running on GCE node
I20260807 19:42:37.081221 2555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:37.082089 2555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:37.083364 2555 hybrid_clock.cc:648] HybridClock initialized: now 1786131757083315 us; error 38 us; skew 500 ppm
I20260807 19:42:37.083696 2555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:37.085906 2555 webserver.cc:460] Webserver started at http://127.14.16.65:44011/ using document root <none> and password file <none>
I20260807 19:42:37.086593 2555 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:37.086709 2555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:37.087033 2555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:37.090193 2555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/data/instance:
uuid: "01869d78cb8c4c26be72fd289ec5ca75"
format_stamp: "Formatted at 2026-08-07 19:42:37 on dist-test-slave-07c2"
server_key: "845097649eed1c334e354cde218bc336"
server_key_iv: "8d58f9049d608a16b84d84cf3d8a8641"
server_key_version: "encryptionkey@0"
I20260807 19:42:37.091039 2555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance:
uuid: "01869d78cb8c4c26be72fd289ec5ca75"
format_stamp: "Formatted at 2026-08-07 19:42:37 on dist-test-slave-07c2"
server_key: "845097649eed1c334e354cde218bc336"
server_key_iv: "8d58f9049d608a16b84d84cf3d8a8641"
server_key_version: "encryptionkey@0"
I20260807 19:42:37.096292 2555 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:42:37.099995 2570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:37.101354 2555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:37.101528 2555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/wal
uuid: "01869d78cb8c4c26be72fd289ec5ca75"
format_stamp: "Formatted at 2026-08-07 19:42:37 on dist-test-slave-07c2"
server_key: "845097649eed1c334e354cde218bc336"
server_key_iv: "8d58f9049d608a16b84d84cf3d8a8641"
server_key_version: "encryptionkey@0"
I20260807 19:42:37.101727 2555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:37.118868 2555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:37.119789 2555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:37.120097 2555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:37.121086 2555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:37.122548 2555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:37.122640 2555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:37.122792 2555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:37.122869 2555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:37.157846 2555 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.65:40399
I20260807 19:42:37.157974 2684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.65:40399 every 8 connection(s)
I20260807 19:42:37.159408 2555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:37.167907 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 2555
I20260807 19:42:37.168148 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:37.168624 14401 external_mini_cluster.cc:1428] Setting key ae7abd4eb4c73619641f66f40ba1e91c
I20260807 19:42:37.171623 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.66:0
--local_ip_for_outbound_sockets=127.14.16.66
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:35391
--builtin_ntp_servers=127.14.16.84:45571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:42:37.172003 2687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.161836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.65:44227 (local address 127.14.16.126:35391)
0807 19:42:37.162317 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.162322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.163403 (+ 1081us) server_negotiation.cc:407] Connection header received
0807 19:42:37.165232 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.165238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.165325 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.165442 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.166480 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.167288 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.169080 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.169621 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.169750 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.171090 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.171095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.171098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:37.171115 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.171160 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:37.171172 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.171247 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.171539 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.171643 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":287,"threads_started":1}
I20260807 19:42:37.173386 2688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.162355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:35391 (local address 127.14.16.65:44227)
0807 19:42:37.163211 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.163247 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:37.164968 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:37.165682 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:37.165690 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:37.165771 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:37.166313 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.166329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.167470 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.167474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.168933 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.168946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.169849 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.169856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.170071 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.170751 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:37.170755 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:37.170808 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:37.170904 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:37.171276 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:37.171313 (+ 37us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:37.171325 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.171532 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.171965 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":266,"threads_started":1}
I20260807 19:42:37.174932 2685 heartbeater.cc:347] Connected to a master server at 127.14.16.126:35391
I20260807 19:42:37.175237 2685 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:37.176040 2685 heartbeater.cc:511] Master 127.14.16.126:35391 requested a full tablet report, sending...
I20260807 19:42:37.178144 2495 ts_manager.cc:194] Registered new tserver with Master: 01869d78cb8c4c26be72fd289ec5ca75 (127.14.16.65:40399)
I20260807 19:42:37.179728 2495 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.16.65:44227
W20260807 19:42:37.515962 2689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:37.516247 2689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:37.516399 2689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:37.523557 2689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:37.523649 2689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:37.523805 2689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.66
I20260807 19:42:37.532785 2689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.16.66
--webserver_port=0
--tserver_master_addrs=127.14.16.126:35391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.66
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:37.534111 2689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:37.535888 2689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:37.549222 2694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:37.549153 2697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:37.549118 2695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:37.550133 2689 server_base.cc:1034] running on GCE node
I20260807 19:42:37.550792 2689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:37.551903 2689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:37.553105 2689 hybrid_clock.cc:648] HybridClock initialized: now 1786131757553060 us; error 44 us; skew 500 ppm
I20260807 19:42:37.553447 2689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:37.555765 2689 webserver.cc:460] Webserver started at http://127.14.16.66:39237/ using document root <none> and password file <none>
I20260807 19:42:37.556452 2689 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:37.556578 2689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:37.556895 2689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:37.559854 2689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/data/instance:
uuid: "9f6e45b791f843f0be4f48ee6240772f"
format_stamp: "Formatted at 2026-08-07 19:42:37 on dist-test-slave-07c2"
server_key: "b0b38e64665687b46b82dd155c994182"
server_key_iv: "34461545719b63af0432695cddffe584"
server_key_version: "encryptionkey@0"
I20260807 19:42:37.560657 2689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance:
uuid: "9f6e45b791f843f0be4f48ee6240772f"
format_stamp: "Formatted at 2026-08-07 19:42:37 on dist-test-slave-07c2"
server_key: "b0b38e64665687b46b82dd155c994182"
server_key_iv: "34461545719b63af0432695cddffe584"
server_key_version: "encryptionkey@0"
I20260807 19:42:37.565712 2689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 19:42:37.569166 2703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:37.570457 2689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:42:37.570618 2689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/wal
uuid: "9f6e45b791f843f0be4f48ee6240772f"
format_stamp: "Formatted at 2026-08-07 19:42:37 on dist-test-slave-07c2"
server_key: "b0b38e64665687b46b82dd155c994182"
server_key_iv: "34461545719b63af0432695cddffe584"
server_key_version: "encryptionkey@0"
I20260807 19:42:37.570834 2689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:37.614337 2689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:37.615196 2689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:37.615550 2689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:37.616578 2689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:37.618108 2689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:37.618239 2689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:37.618360 2689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:37.618445 2689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:37.653144 2689 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.66:35985
I20260807 19:42:37.653231 2816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.66:35985 every 8 connection(s)
I20260807 19:42:37.654633 2689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:37.654911 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 2689
I20260807 19:42:37.655092 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:37.655714 14401 external_mini_cluster.cc:1428] Setting key 9a99a44e4c7cad9e41a8f73f76b36ba8
I20260807 19:42:37.659111 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.16.67:0
--local_ip_for_outbound_sockets=127.14.16.67
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:35391
--builtin_ntp_servers=127.14.16.84:45571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:42:37.668368 2687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.657489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:40361 (local address 127.14.16.126:35391)
0807 19:42:37.657689 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.657697 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.659330 (+ 1633us) server_negotiation.cc:407] Connection header received
0807 19:42:37.661328 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.661336 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.661430 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.661539 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.662590 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.663746 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.665238 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.665934 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.666076 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.667433 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.667438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.667442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:37.667461 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.667498 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:37.667510 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.667607 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.667874 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.667992 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:42:37.669389 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.658112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:35391 (local address 127.14.16.66:40361)
0807 19:42:37.659127 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.659168 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:37.661062 (+ 1894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:37.661737 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:37.661745 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:37.661842 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:37.662421 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.662440 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.663890 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.663894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.665090 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.665103 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.666134 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.666139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.666337 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.667067 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:37.667070 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:37.667129 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:37.667228 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:37.667606 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:37.667619 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:37.667629 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.667850 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.668198 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":382,"threads_started":1}
I20260807 19:42:37.670838 2817 heartbeater.cc:347] Connected to a master server at 127.14.16.126:35391
I20260807 19:42:37.671164 2817 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:37.672119 2817 heartbeater.cc:511] Master 127.14.16.126:35391 requested a full tablet report, sending...
I20260807 19:42:37.673748 2495 ts_manager.cc:194] Registered new tserver with Master: 9f6e45b791f843f0be4f48ee6240772f (127.14.16.66:35985)
I20260807 19:42:37.675060 2495 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.16.66:40361
W20260807 19:42:38.003520 2820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:38.003808 2820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:38.003957 2820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:38.011258 2820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:38.011389 2820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:38.011562 2820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.16.67
I20260807 19:42:38.020495 2820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:45571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.16.67
--webserver_port=0
--tserver_master_addrs=127.14.16.126:35391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.16.67
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:38.021867 2820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:38.023622 2820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:38.036456 2825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:38.036389 2826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:38.036432 2828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:38.037106 2820 server_base.cc:1034] running on GCE node
I20260807 19:42:38.037719 2820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:38.038637 2820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:38.039860 2820 hybrid_clock.cc:648] HybridClock initialized: now 1786131758039813 us; error 40 us; skew 500 ppm
I20260807 19:42:38.040165 2820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:38.042390 2820 webserver.cc:460] Webserver started at http://127.14.16.67:36271/ using document root <none> and password file <none>
I20260807 19:42:38.043085 2820 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:38.043213 2820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:38.043572 2820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:38.046454 2820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/data/instance:
uuid: "818e0252f78c4820b63632fccffeea97"
format_stamp: "Formatted at 2026-08-07 19:42:38 on dist-test-slave-07c2"
server_key: "e1cb185fa94983ade782afae00608a72"
server_key_iv: "b42443b5966d8a0e9a82772532633f25"
server_key_version: "encryptionkey@0"
I20260807 19:42:38.047235 2820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance:
uuid: "818e0252f78c4820b63632fccffeea97"
format_stamp: "Formatted at 2026-08-07 19:42:38 on dist-test-slave-07c2"
server_key: "e1cb185fa94983ade782afae00608a72"
server_key_iv: "b42443b5966d8a0e9a82772532633f25"
server_key_version: "encryptionkey@0"
I20260807 19:42:38.052594 2820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:42:38.056079 2834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:38.057350 2820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:38.057524 2820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/wal
uuid: "818e0252f78c4820b63632fccffeea97"
format_stamp: "Formatted at 2026-08-07 19:42:38 on dist-test-slave-07c2"
server_key: "e1cb185fa94983ade782afae00608a72"
server_key_iv: "b42443b5966d8a0e9a82772532633f25"
server_key_version: "encryptionkey@0"
I20260807 19:42:38.057730 2820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:38.068713 2820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:38.069466 2820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:38.069768 2820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:38.070685 2820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:38.072126 2820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:38.072245 2820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:38.072374 2820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:38.072458 2820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:38.107144 2820 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.67:44427
I20260807 19:42:38.107237 2946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.67:44427 every 8 connection(s)
I20260807 19:42:38.108562 2820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:38.113147 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 2820
I20260807 19:42:38.113353 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:38.113819 14401 external_mini_cluster.cc:1428] Setting key cbe132758363a987cda885842a4aa058
I20260807 19:42:38.121534 2687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.111203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.67:38265 (local address 127.14.16.126:35391)
0807 19:42:38.111455 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.111460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.112815 (+ 1355us) server_negotiation.cc:407] Connection header received
0807 19:42:38.114889 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.114894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.115008 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.115123 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:38.116504 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.117489 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.118994 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.119525 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.119630 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.120886 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:38.120891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:38.120894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:38.120912 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:38.120945 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:38.120957 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:38.121038 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.121224 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.121304 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 19:42:38.122517 2949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.111755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:35391 (local address 127.14.16.67:38265)
0807 19:42:38.112648 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:38.112684 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:38.114598 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:38.115394 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:38.115402 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:38.115518 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:38.116317 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.116342 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.117636 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.117640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.118856 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.118869 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.119713 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.119718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.119921 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.120576 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:38.120579 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:38.120615 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:38.120691 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:38.121048 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:38.121055 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:38.121061 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:42:38.121196 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 19:42:38.121480 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":274,"threads_started":1}
I20260807 19:42:38.123900 2947 heartbeater.cc:347] Connected to a master server at 127.14.16.126:35391
I20260807 19:42:38.124176 2947 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:38.124863 2947 heartbeater.cc:511] Master 127.14.16.126:35391 requested a full tablet report, sending...
I20260807 19:42:38.126130 2495 ts_manager.cc:194] Registered new tserver with Master: 818e0252f78c4820b63632fccffeea97 (127.14.16.67:44427)
I20260807 19:42:38.126922 2495 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.16.67:38265
I20260807 19:42:38.129156 14401 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:42:38.138979 2687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.133371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54374 (local address 127.14.16.126:35391)
0807 19:42:38.133532 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.133536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.134007 (+ 471us) server_negotiation.cc:407] Connection header received
0807 19:42:38.134148 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.134152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.134221 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.134303 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:38.134759 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.135487 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.136820 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.137221 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.137310 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.138439 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:38.138443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:38.138446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:38.138463 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:38.138492 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:38.138503 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:38.138575 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.138693 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.138762 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 19:42:38.142665 2495 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:38.150484 2495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:38.182631 2685 heartbeater.cc:503] Master 127.14.16.126:35391 was elected leader, sending a full tablet report...
I20260807 19:42:38.183157 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.167837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.66:35985 (local address 127.0.0.1:50856)
0807 19:42:38.171002 (+ 3165us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:38.171016 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:38.171157 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:38.172514 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:38.172525 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:38.172568 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:38.173101 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.173125 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.176254 (+ 3129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.176263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.179499 (+ 3236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.179516 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.182105 (+ 2589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.182112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.182357 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.182411 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 19:42:38.182537 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 19:42:38.182733 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3069,"thread_start_us":1020,"threads_started":1}
I20260807 19:42:38.184325 2962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.168130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50856 (local address 127.14.16.66:35985)
0807 19:42:38.170552 (+ 2422us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.170559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.171121 (+ 562us) server_negotiation.cc:407] Connection header received
0807 19:42:38.171650 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.171662 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.171983 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.173331 (+ 1348us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:38.173596 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.176061 (+ 2465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.179713 (+ 3652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.181438 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.181716 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.181979 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.183751 (+ 1772us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.183958 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2270,"thread_start_us":2149,"threads_started":1}
I20260807 19:42:38.188102 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.170826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.14.16.67:44427)
0807 19:42:38.172557 (+ 1731us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.172570 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.179915 (+ 7345us) server_negotiation.cc:407] Connection header received
0807 19:42:38.180097 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.180105 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.180319 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.180502 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:38.181329 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.182565 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.185709 (+ 3144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.186891 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.187052 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.187383 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.187529 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.187703 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1503,"thread_start_us":521,"threads_started":1}
I20260807 19:42:38.188427 2961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.167741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:40399 (local address 127.0.0.1:49794)
0807 19:42:38.169395 (+ 1654us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:38.169457 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:38.169672 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:38.180311 (+ 10639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:38.180317 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:38.180345 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:38.180750 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.180765 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.182940 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.182945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.185720 (+ 2775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.185736 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.187625 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.187632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.187803 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.187837 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:42:38.187914 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 19:42:38.188049 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1274,"mutex_wait_us":93,"thread_start_us":12291,"threads_started":1}
I20260807 19:42:38.188052 2960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.170439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:44427 (local address 127.0.0.1:39776)
0807 19:42:38.179783 (+ 9344us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:38.179805 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:38.179938 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:38.180631 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:38.180635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:38.180663 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:38.181066 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.181082 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.182892 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.182900 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.185466 (+ 2566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.185486 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.187086 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.187092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.187275 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.187361 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 19:42:38.187471 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 19:42:38.187618 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9206,"mutex_wait_us":32,"thread_start_us":341,"threads_started":1}
I20260807 19:42:38.189246 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.168491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49794 (local address 127.14.16.65:40399)
0807 19:42:38.179666 (+ 11175us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.179675 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.179706 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:42:38.179808 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.179813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.180061 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.180326 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:38.180983 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.182750 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.185932 (+ 3182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.187388 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.188340 (+ 952us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.188613 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.188688 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.188850 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10987,"thread_start_us":10814,"threads_started":1}
I20260807 19:42:38.188575 2751 tablet_service.cc:1467] Processing CreateTablet for tablet c16b1cf5a51d4cb9922f92fcea81e9a6 (DEFAULT_TABLE table=test-table [id=8f24be5924ef41cf9e099c400506cfb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:38.190421 2751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c16b1cf5a51d4cb9922f92fcea81e9a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:38.192287 2882 tablet_service.cc:1467] Processing CreateTablet for tablet c16b1cf5a51d4cb9922f92fcea81e9a6 (DEFAULT_TABLE table=test-table [id=8f24be5924ef41cf9e099c400506cfb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:38.193537 2882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c16b1cf5a51d4cb9922f92fcea81e9a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:38.193142 2619 tablet_service.cc:1467] Processing CreateTablet for tablet c16b1cf5a51d4cb9922f92fcea81e9a6 (DEFAULT_TABLE table=test-table [id=8f24be5924ef41cf9e099c400506cfb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:38.194478 2619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c16b1cf5a51d4cb9922f92fcea81e9a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:38.207240 2966 tablet_bootstrap.cc:492] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97: Bootstrap starting.
I20260807 19:42:38.209659 2968 tablet_bootstrap.cc:492] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75: Bootstrap starting.
I20260807 19:42:38.210831 2967 tablet_bootstrap.cc:492] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f: Bootstrap starting.
I20260807 19:42:38.211850 2966 tablet_bootstrap.cc:654] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:38.213253 2966 log.cc:824] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:38.213802 2968 tablet_bootstrap.cc:654] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:38.215003 2967 tablet_bootstrap.cc:654] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:38.215056 2968 log.cc:824] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:38.216233 2967 log.cc:824] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:38.216733 2966 tablet_bootstrap.cc:492] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97: No bootstrap required, opened a new log
I20260807 19:42:38.217022 2966 ts_tablet_manager.cc:1397] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 19:42:38.218474 2968 tablet_bootstrap.cc:492] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75: No bootstrap required, opened a new log
I20260807 19:42:38.218770 2968 ts_tablet_manager.cc:1397] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:42:38.225450 2967 tablet_bootstrap.cc:492] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f: No bootstrap required, opened a new log
I20260807 19:42:38.225724 2967 ts_tablet_manager.cc:1397] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260807 19:42:38.229061 2966 raft_consensus.cc:348] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01869d78cb8c4c26be72fd289ec5ca75" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 40399 } } peers { permanent_uuid: "9f6e45b791f843f0be4f48ee6240772f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35985 } } peers { permanent_uuid: "818e0252f78c4820b63632fccffeea97" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44427 } }
I20260807 19:42:38.229584 2966 raft_consensus.cc:374] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:38.229701 2966 raft_consensus.cc:729] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 818e0252f78c4820b63632fccffeea97, State: Initialized, Role: FOLLOWER
I20260807 19:42:38.230585 2966 consensus_queue.cc:260] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01869d78cb8c4c26be72fd289ec5ca75" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 40399 } } peers { permanent_uuid: "9f6e45b791f843f0be4f48ee6240772f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35985 } } peers { permanent_uuid: "818e0252f78c4820b63632fccffeea97" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44427 } }
I20260807 19:42:38.232012 2947 heartbeater.cc:503] Master 127.14.16.126:35391 was elected leader, sending a full tablet report...
I20260807 19:42:38.232741 2966 ts_tablet_manager.cc:1428] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
I20260807 19:42:38.231451 2968 raft_consensus.cc:348] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01869d78cb8c4c26be72fd289ec5ca75" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 40399 } } peers { permanent_uuid: "9f6e45b791f843f0be4f48ee6240772f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35985 } } peers { permanent_uuid: "818e0252f78c4820b63632fccffeea97" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44427 } }
I20260807 19:42:38.233127 2968 raft_consensus.cc:374] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:38.233237 2968 raft_consensus.cc:729] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01869d78cb8c4c26be72fd289ec5ca75, State: Initialized, Role: FOLLOWER
I20260807 19:42:38.233862 2968 consensus_queue.cc:260] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01869d78cb8c4c26be72fd289ec5ca75" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 40399 } } peers { permanent_uuid: "9f6e45b791f843f0be4f48ee6240772f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35985 } } peers { permanent_uuid: "818e0252f78c4820b63632fccffeea97" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44427 } }
I20260807 19:42:38.233807 2967 raft_consensus.cc:348] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01869d78cb8c4c26be72fd289ec5ca75" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 40399 } } peers { permanent_uuid: "9f6e45b791f843f0be4f48ee6240772f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35985 } } peers { permanent_uuid: "818e0252f78c4820b63632fccffeea97" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44427 } }
I20260807 19:42:38.234203 2967 raft_consensus.cc:374] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:38.234338 2967 raft_consensus.cc:729] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f6e45b791f843f0be4f48ee6240772f, State: Initialized, Role: FOLLOWER
I20260807 19:42:38.235066 2967 consensus_queue.cc:260] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01869d78cb8c4c26be72fd289ec5ca75" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 40399 } } peers { permanent_uuid: "9f6e45b791f843f0be4f48ee6240772f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35985 } } peers { permanent_uuid: "818e0252f78c4820b63632fccffeea97" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44427 } }
I20260807 19:42:38.235791 2968 ts_tablet_manager.cc:1428] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260807 19:42:38.237578 2817 heartbeater.cc:503] Master 127.14.16.126:35391 was elected leader, sending a full tablet report...
I20260807 19:42:38.238200 2967 ts_tablet_manager.cc:1428] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
W20260807 19:42:38.360287 2948 tablet.cc:2369] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:42:38.406961 2818 tablet.cc:2369] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:42:38.413884 2686 tablet.cc:2369] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:38.575951 2974 raft_consensus.cc:482] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:38.576315 2974 raft_consensus.cc:504] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01869d78cb8c4c26be72fd289ec5ca75" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 40399 } } peers { permanent_uuid: "9f6e45b791f843f0be4f48ee6240772f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35985 } } peers { permanent_uuid: "818e0252f78c4820b63632fccffeea97" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44427 } }
I20260807 19:42:38.577991 2974 leader_election.cc:288] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01869d78cb8c4c26be72fd289ec5ca75 (127.14.16.65:40399), 818e0252f78c4820b63632fccffeea97 (127.14.16.67:44427)
I20260807 19:42:38.585192 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.578441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:59253 (local address 127.14.16.67:44427)
0807 19:42:38.578629 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.578635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.579411 (+ 776us) server_negotiation.cc:407] Connection header received
0807 19:42:38.579828 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.579836 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.579931 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.580026 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:38.580689 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.581515 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.583494 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.584244 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.584368 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.584453 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.584789 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.584922 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:42:38.585198 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.578441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.16.66:58027 (local address 127.14.16.65:40399)
0807 19:42:38.578629 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.578635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.579930 (+ 1295us) server_negotiation.cc:407] Connection header received
0807 19:42:38.580089 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.580093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.580189 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.580296 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:38.580950 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.581661 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.583558 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.584340 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.584510 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.584578 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.584852 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.584977 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:42:38.585213 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.578505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.67:44427 (local address 127.14.16.66:59253)
0807 19:42:38.579229 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:38.579245 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:38.579569 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:38.580076 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:38.580080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:38.580107 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:38.580537 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.580547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.581648 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.581653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.583257 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.583269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.584443 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.584450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.584624 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.584657 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:42:38.584729 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 19:42:38.584856 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":1,"thread_start_us":429,"threads_started":1}
I20260807 19:42:38.585338 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.578505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.65:40399 (local address 127.14.16.66:58027)
0807 19:42:38.579810 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:38.579829 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:38.579952 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:38.580320 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:38.580323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:38.580342 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:38.580768 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.580780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.581817 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.581824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.583265 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.583277 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.584548 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.584553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.584680 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.584701 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 19:42:38.584812 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 19:42:38.584931 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1170,"mutex_wait_us":46,"spinlock_wait_cycles":23936,"thread_start_us":893,"threads_started":1}
I20260807 19:42:38.586401 2902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16b1cf5a51d4cb9922f92fcea81e9a6" candidate_uuid: "9f6e45b791f843f0be4f48ee6240772f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "818e0252f78c4820b63632fccffeea97" is_pre_election: true
I20260807 19:42:38.586404 2639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16b1cf5a51d4cb9922f92fcea81e9a6" candidate_uuid: "9f6e45b791f843f0be4f48ee6240772f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01869d78cb8c4c26be72fd289ec5ca75" is_pre_election: true
I20260807 19:42:38.586884 2902 raft_consensus.cc:2454] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f6e45b791f843f0be4f48ee6240772f in term 0.
I20260807 19:42:38.586884 2639 raft_consensus.cc:2454] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f6e45b791f843f0be4f48ee6240772f in term 0.
I20260807 19:42:38.587524 2705 leader_election.cc:302] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 01869d78cb8c4c26be72fd289ec5ca75, 818e0252f78c4820b63632fccffeea97, 9f6e45b791f843f0be4f48ee6240772f; no voters:
I20260807 19:42:38.587924 2974 raft_consensus.cc:2781] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:38.588079 2974 raft_consensus.cc:482] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:38.588189 2974 raft_consensus.cc:3037] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:38.590327 2974 raft_consensus.cc:504] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01869d78cb8c4c26be72fd289ec5ca75" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 40399 } } peers { permanent_uuid: "9f6e45b791f843f0be4f48ee6240772f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35985 } } peers { permanent_uuid: "818e0252f78c4820b63632fccffeea97" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44427 } }
I20260807 19:42:38.590935 2974 leader_election.cc:288] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [CANDIDATE]: Term 1 election: Requested vote from peers 01869d78cb8c4c26be72fd289ec5ca75 (127.14.16.65:40399), 818e0252f78c4820b63632fccffeea97 (127.14.16.67:44427)
I20260807 19:42:38.591423 2639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16b1cf5a51d4cb9922f92fcea81e9a6" candidate_uuid: "9f6e45b791f843f0be4f48ee6240772f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01869d78cb8c4c26be72fd289ec5ca75"
I20260807 19:42:38.591424 2902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16b1cf5a51d4cb9922f92fcea81e9a6" candidate_uuid: "9f6e45b791f843f0be4f48ee6240772f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "818e0252f78c4820b63632fccffeea97"
I20260807 19:42:38.591638 2639 raft_consensus.cc:3037] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:38.591638 2902 raft_consensus.cc:3037] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:38.594933 2902 raft_consensus.cc:2454] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f6e45b791f843f0be4f48ee6240772f in term 1.
I20260807 19:42:38.594933 2639 raft_consensus.cc:2454] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f6e45b791f843f0be4f48ee6240772f in term 1.
I20260807 19:42:38.595441 2704 leader_election.cc:302] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 01869d78cb8c4c26be72fd289ec5ca75, 818e0252f78c4820b63632fccffeea97, 9f6e45b791f843f0be4f48ee6240772f; no voters:
I20260807 19:42:38.595816 2974 raft_consensus.cc:2781] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:38.596289 2974 raft_consensus.cc:686] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 1 LEADER]: Becoming Leader. State: Replica: 9f6e45b791f843f0be4f48ee6240772f, State: Running, Role: LEADER
I20260807 19:42:38.596797 2974 consensus_queue.cc:237] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01869d78cb8c4c26be72fd289ec5ca75" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 40399 } } peers { permanent_uuid: "9f6e45b791f843f0be4f48ee6240772f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35985 } } peers { permanent_uuid: "818e0252f78c4820b63632fccffeea97" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44427 } }
I20260807 19:42:38.601876 2494 catalog_manager.cc:5495] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f6e45b791f843f0be4f48ee6240772f (127.14.16.66). New cstate: current_term: 1 leader_uuid: "9f6e45b791f843f0be4f48ee6240772f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01869d78cb8c4c26be72fd289ec5ca75" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 40399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f6e45b791f843f0be4f48ee6240772f" member_type: VOTER last_known_addr { host: "127.14.16.66" port: 35985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "818e0252f78c4820b63632fccffeea97" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44427 } health_report { overall_health: UNKNOWN } } }
I20260807 19:42:38.633077 2962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.627829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57358 (local address 127.14.16.66:35985)
0807 19:42:38.628038 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.628044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.628064 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:42:38.628221 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.628225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.628325 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.628479 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:38.628866 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.629575 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.631019 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.631558 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.631690 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.632102 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:38.632608 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:38.632717 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.632815 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.632878 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:42:38.640990 2902 raft_consensus.cc:1261] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97 [term 1 FOLLOWER]: Refusing update from remote peer 9f6e45b791f843f0be4f48ee6240772f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:38.640991 2639 raft_consensus.cc:1261] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75 [term 1 FOLLOWER]: Refusing update from remote peer 9f6e45b791f843f0be4f48ee6240772f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:38.642064 2977 consensus_queue.cc:1035] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01869d78cb8c4c26be72fd289ec5ca75" member_type: VOTER last_known_addr { host: "127.14.16.65" port: 40399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:38.642423 2974 consensus_queue.cc:1035] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [LEADER]: Connected to new peer: Peer: permanent_uuid: "818e0252f78c4820b63632fccffeea97" member_type: VOTER last_known_addr { host: "127.14.16.67" port: 44427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:38.667811 2495 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:38.668087 2495 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:38.672051 2495 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d18ba4309bd94748b938bb5139f1b4de: Sending DeleteTablet for 3 replicas of tablet c16b1cf5a51d4cb9922f92fcea81e9a6
I20260807 19:42:38.673703 2619 tablet_service.cc:1514] Processing DeleteTablet for tablet c16b1cf5a51d4cb9922f92fcea81e9a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:38 UTC) from {username='slave'} at 127.0.0.1:49794
I20260807 19:42:38.674435 2751 tablet_service.cc:1514] Processing DeleteTablet for tablet c16b1cf5a51d4cb9922f92fcea81e9a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:38 UTC) from {username='slave'} at 127.0.0.1:50856
I20260807 19:42:38.675155 2989 tablet_replica.cc:330] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75: stopping tablet replica
I20260807 19:42:38.675308 2882 tablet_service.cc:1514] Processing DeleteTablet for tablet c16b1cf5a51d4cb9922f92fcea81e9a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:38 UTC) from {username='slave'} at 127.0.0.1:39776
I20260807 19:42:38.675639 2989 raft_consensus.cc:2229] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:38.676048 2989 raft_consensus.cc:2258] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:38.676448 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 2555
I20260807 19:42:38.686956 2989 ts_tablet_manager.cc:1905] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:38.691032 2991 tablet_replica.cc:330] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97: stopping tablet replica
I20260807 19:42:38.692209 2990 tablet_replica.cc:330] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f: stopping tablet replica
I20260807 19:42:38.692605 2990 raft_consensus.cc:2229] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:38.693288 2990 raft_consensus.cc:2258] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:38.694267 2991 raft_consensus.cc:2229] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:38.694342 2989 ts_tablet_manager.cc:1918] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:38.694499 2989 log.cc:1196] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-0/wal/wals/c16b1cf5a51d4cb9922f92fcea81e9a6
I20260807 19:42:38.694799 2991 raft_consensus.cc:2258] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:38.695029 2989 ts_tablet_manager.cc:1939] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 01869d78cb8c4c26be72fd289ec5ca75: Deleting consensus metadata
I20260807 19:42:38.697094 2991 ts_tablet_manager.cc:1905] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:38.697106 2990 ts_tablet_manager.cc:1905] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:38.699577 2477 catalog_manager.cc:4841] TS 01869d78cb8c4c26be72fd289ec5ca75 (127.14.16.65:40399): tablet c16b1cf5a51d4cb9922f92fcea81e9a6 (table test-table [id=8f24be5924ef41cf9e099c400506cfb5]) successfully deleted
I20260807 19:42:38.704141 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.678406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57636 (local address 127.14.16.65:40399)
0807 19:42:38.678619 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.678627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.684879 (+ 6252us) server_negotiation.cc:407] Connection header received
0807 19:42:38.685837 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.685845 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.685966 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.686057 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:38.687610 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.689243 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.692118 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.692810 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.692941 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.702064 (+ 9123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:38.702075 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:38.702091 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:38.702826 (+ 735us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:38.702886 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:38.702912 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:38.703027 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.703640 (+ 613us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.703776 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:42:38.704885 2680 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:38.704869 2991 ts_tablet_manager.cc:1918] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:38.705034 2991 log.cc:1196] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-2/wal/wals/c16b1cf5a51d4cb9922f92fcea81e9a6
I20260807 19:42:38.705777 2991 ts_tablet_manager.cc:1939] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 818e0252f78c4820b63632fccffeea97: Deleting consensus metadata
I20260807 19:42:38.706182 2990 ts_tablet_manager.cc:1918] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:38.706393 2990 log.cc:1196] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f: Deleting WAL directory at /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131435615921-14401-0/minicluster-data/ts-1/wal/wals/c16b1cf5a51d4cb9922f92fcea81e9a6
I20260807 19:42:38.707069 2990 ts_tablet_manager.cc:1939] T c16b1cf5a51d4cb9922f92fcea81e9a6 P 9f6e45b791f843f0be4f48ee6240772f: Deleting consensus metadata
I20260807 19:42:38.707969 2478 catalog_manager.cc:4841] TS 818e0252f78c4820b63632fccffeea97 (127.14.16.67:44427): tablet c16b1cf5a51d4cb9922f92fcea81e9a6 (table test-table [id=8f24be5924ef41cf9e099c400506cfb5]) successfully deleted
I20260807 19:42:38.708853 2480 catalog_manager.cc:4841] TS 9f6e45b791f843f0be4f48ee6240772f (127.14.16.66:35985): tablet c16b1cf5a51d4cb9922f92fcea81e9a6 (table test-table [id=8f24be5924ef41cf9e099c400506cfb5]) successfully deleted
I20260807 19:42:38.797250 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 2555
I20260807 19:42:38.817503 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 2689
I20260807 19:42:38.824748 2962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.818361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57368 (local address 127.14.16.66:35985)
0807 19:42:38.818546 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.818554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.818574 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:42:38.818790 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.818796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.818900 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.819009 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:38.819489 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.820267 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.821540 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.821992 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.822125 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.823245 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:38.823254 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:38.823268 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:38.823943 (+ 675us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:38.823995 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:38.824016 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:38.824145 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.824280 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.824379 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:42:38.825399 2812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:38.938983 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 2689
I20260807 19:42:38.960255 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 2820
I20260807 19:42:38.966838 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.961127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51492 (local address 127.14.16.67:44427)
0807 19:42:38.961328 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.961333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.961355 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:42:38.961526 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.961530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.961612 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.961711 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:38.962095 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.962862 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.964101 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.964487 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.964576 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.965551 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:38.965559 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:38.965573 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:38.966126 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:38.966178 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:38.966197 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:38.966281 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.966390 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.966475 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 19:42:38.967545 2942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:39.055110 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 2820
I20260807 19:42:39.074465 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 2461
I20260807 19:42:39.075699 2525 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:39.185778 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 2461
2026-08-07T19:42:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3127 ms)
[----------] 6 tests from EncryptionPolicyTest (22612 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:39 dist-test-slave-07c2 krb5kdc[2999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:39 dist-test-slave-07c2 krb5kdc[2999](info): set up 2 sockets
Aug 07 19:42:39 dist-test-slave-07c2 krb5kdc[2999](info): commencing operation
krb5kdc: starting...
W20260807 19:42:40.350463 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:40 dist-test-slave-07c2 krb5kdc[2999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:42:40.677017 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44173
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:41.015182 3012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:41.015574 3012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:41.015740 3012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:41.022945 3012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:41.023098 3012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:41.023208 3012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:41.023336 3012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:41.023408 3012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:41.033317 3012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44173
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:41.034596 3012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:41.035945 3012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:41.047942 3017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:41.047967 3018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:41.047875 3020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:41.049038 3012 server_base.cc:1034] running on GCE node
I20260807 19:42:41.049885 3012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:41.051402 3012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:41.052672 3012 hybrid_clock.cc:648] HybridClock initialized: now 1786131761052601 us; error 75 us; skew 500 ppm
I20260807 19:42:41.053040 3012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:41 dist-test-slave-07c2 krb5kdc[2999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131761, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:41.056284 3012 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:42:41.057703 3012 webserver.cc:460] Webserver started at http://127.0.0.1:44245/ using document root <none> and password file <none>
I20260807 19:42:41.058368 3012 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:41.058508 3012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:41.058841 3012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:41.061915 3012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "26c67d922f67413ba05800ff7cc6fdb3"
format_stamp: "Formatted at 2026-08-07 19:42:41 on dist-test-slave-07c2"
server_key: "4b74729edc17bc6b9e3a91d21c482a15"
server_key_iv: "368c28a3ed55e82bb6eb700b63b245e0"
server_key_version: "encryptionkey@0"
I20260807 19:42:41.062700 3012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "26c67d922f67413ba05800ff7cc6fdb3"
format_stamp: "Formatted at 2026-08-07 19:42:41 on dist-test-slave-07c2"
server_key: "4b74729edc17bc6b9e3a91d21c482a15"
server_key_iv: "368c28a3ed55e82bb6eb700b63b245e0"
server_key_version: "encryptionkey@0"
I20260807 19:42:41.067890 3012 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 19:42:41.071350 3027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:41.072566 3012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:41.072757 3012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "26c67d922f67413ba05800ff7cc6fdb3"
format_stamp: "Formatted at 2026-08-07 19:42:41 on dist-test-slave-07c2"
server_key: "4b74729edc17bc6b9e3a91d21c482a15"
server_key_iv: "368c28a3ed55e82bb6eb700b63b245e0"
server_key_version: "encryptionkey@0"
I20260807 19:42:41.072947 3012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:41.102470 3012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:41.106951 3012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:41.107331 3012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:41.126603 3012 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44173
I20260807 19:42:41.126716 3079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44173 every 8 connection(s)
I20260807 19:42:41.128100 3012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:41.131953 3080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:41.136219 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 3012
I20260807 19:42:41.136456 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:41.137287 14401 external_mini_cluster.cc:1428] Setting key 615e58b4f63d9641b410bbf83662003f
I20260807 19:42:41.143781 3080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3: Bootstrap starting.
I20260807 19:42:41.146888 3080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:41.148351 3080 log.cc:824] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:41.151212 3080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3: No bootstrap required, opened a new log
Aug 07 19:42:41 dist-test-slave-07c2 krb5kdc[2999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:42:41.160841 3080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c67d922f67413ba05800ff7cc6fdb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44173 } }
I20260807 19:42:41.161295 3080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:41.161409 3080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c67d922f67413ba05800ff7cc6fdb3, State: Initialized, Role: FOLLOWER
I20260807 19:42:41.162074 3080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c67d922f67413ba05800ff7cc6fdb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44173 } }
I20260807 19:42:41.162212 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.139429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53089 (local address 127.0.0.1:44173)
0807 19:42:41.140289 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:41.140300 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:41.140343 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:42:41.142481 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:41.142504 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:41.142877 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:41.143269 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:41.144874 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.146216 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.147904 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.148833 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.149032 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:41.153908 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:41.153917 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:41.153935 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:41.153976 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:41.157333 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:41.158438 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:41.158446 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:41.158456 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:41.158649 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:41.159173 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:41.159177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:41.159179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:41.159841 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:41.160078 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:41.160606 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 19:42:41.160978 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":394,"threads_started":1}
I20260807 19:42:41.162376 3080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:41.162554 3080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:41.162740 3080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:41.165470 3080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c67d922f67413ba05800ff7cc6fdb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44173 } }
I20260807 19:42:41.166013 3080 leader_election.cc:302] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26c67d922f67413ba05800ff7cc6fdb3; no voters:
I20260807 19:42:41.166627 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:41.166676 3080 leader_election.cc:288] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:41.166800 3085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:41.168891 3080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:41.171607 3085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [term 1 LEADER]: Becoming Leader. State: Replica: 26c67d922f67413ba05800ff7cc6fdb3, State: Running, Role: LEADER
I20260807 19:42:41.172286 3085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c67d922f67413ba05800ff7cc6fdb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44173 } }
I20260807 19:42:41.183452 3087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c67d922f67413ba05800ff7cc6fdb3. Latest consensus state: current_term: 1 leader_uuid: "26c67d922f67413ba05800ff7cc6fdb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c67d922f67413ba05800ff7cc6fdb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44173 } } }
I20260807 19:42:41.184031 3087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:41.190435 3086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26c67d922f67413ba05800ff7cc6fdb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c67d922f67413ba05800ff7cc6fdb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44173 } } }
I20260807 19:42:41.190789 3086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:41.190975 3101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:41.196983 3101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:41.212231 3101 catalog_manager.cc:1347] Generated new cluster ID: d68ed1db36664cf89f13ee891fbe93d1
I20260807 19:42:41.212384 3101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:41.213297 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.189999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43245 (local address 127.0.0.1:44173)
0807 19:42:41.190256 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:41.190265 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:41.192043 (+ 1778us) server_negotiation.cc:407] Connection header received
0807 19:42:41.192308 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:41.192318 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:41.192435 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:41.192560 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:41.194604 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.196458 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.199018 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.200097 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.200249 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:41.204769 (+ 4520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:41.204780 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:41.204787 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:41.204828 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:41.209124 (+ 4296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:41.210308 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:41.210314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:41.210318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:41.210483 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:41.211123 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:41.211128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:41.211132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:41.211579 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:41.211831 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:41.212683 (+ 852us) server_negotiation.cc:299] Negotiation successful
0807 19:42:41.212896 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:42:41.229281 3101 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:41.231180 3101 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:41.240667 3101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26c67d922f67413ba05800ff7cc6fdb3: Generated new TSK 0
I20260807 19:42:41.241910 3101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:41.253296 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3012
I20260807 19:42:41.255336 3075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:41.347436 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 3012
W20260807 19:42:41.368162 2997 connection.cc:535] client connection to 127.0.0.1:44173 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44173): Connection reset by peer (error 104)
2026-08-07T19:42:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2153 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:41 dist-test-slave-07c2 krb5kdc[3114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:41 dist-test-slave-07c2 krb5kdc[3114](info): set up 2 sockets
Aug 07 19:42:41 dist-test-slave-07c2 krb5kdc[3114](info): commencing operation
krb5kdc: starting...
W20260807 19:42:42.519595 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:42 dist-test-slave-07c2 krb5kdc[3114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:42:42.846823 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:43.194572 3127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:43.194955 3127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:43.195117 3127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:43.202708 3127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:43.202870 3127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:43.202975 3127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:43.203078 3127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:43.203153 3127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:43.213555 3127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33555
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:43.214944 3127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:43.216338 3127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:43.228188 3133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:43.228246 3135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:43.228478 3132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:43.229415 3127 server_base.cc:1034] running on GCE node
I20260807 19:42:43.230401 3127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:43.232115 3127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:43.233314 3127 hybrid_clock.cc:648] HybridClock initialized: now 1786131763233276 us; error 45 us; skew 500 ppm
I20260807 19:42:43.233618 3127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:43 dist-test-slave-07c2 krb5kdc[3114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131763, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:43.236707 3127 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:42:43.238154 3127 webserver.cc:460] Webserver started at http://127.0.0.1:38717/ using document root <none> and password file <none>
I20260807 19:42:43.238798 3127 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:43.238901 3127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:43.239162 3127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:43.242229 3127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "7f3d99f73f794920ad231e73f8c75d75"
format_stamp: "Formatted at 2026-08-07 19:42:43 on dist-test-slave-07c2"
server_key: "f90b3a9349f35a7835435df8703703f8"
server_key_iv: "6298493529b6bec4193b4ecb6f85394f"
server_key_version: "encryptionkey@0"
I20260807 19:42:43.242959 3127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "7f3d99f73f794920ad231e73f8c75d75"
format_stamp: "Formatted at 2026-08-07 19:42:43 on dist-test-slave-07c2"
server_key: "f90b3a9349f35a7835435df8703703f8"
server_key_iv: "6298493529b6bec4193b4ecb6f85394f"
server_key_version: "encryptionkey@0"
I20260807 19:42:43.248504 3127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 19:42:43.251904 3142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:43.253222 3127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:43.253371 3127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "7f3d99f73f794920ad231e73f8c75d75"
format_stamp: "Formatted at 2026-08-07 19:42:43 on dist-test-slave-07c2"
server_key: "f90b3a9349f35a7835435df8703703f8"
server_key_iv: "6298493529b6bec4193b4ecb6f85394f"
server_key_version: "encryptionkey@0"
I20260807 19:42:43.253542 3127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:43.271191 3127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:43.275491 3127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:43.275743 3127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:43.294909 3127 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33555
I20260807 19:42:43.294947 3195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33555 every 8 connection(s)
I20260807 19:42:43.296401 3127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:43.299517 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 3127
I20260807 19:42:43.299707 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:43.300271 14401 external_mini_cluster.cc:1428] Setting key d32110b963d970521f6977d25a1d29d2
I20260807 19:42:43.300647 3196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:43.313668 3196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75: Bootstrap starting.
I20260807 19:42:43.314147 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:43.302128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36769 (local address 127.0.0.1:33555)
0807 19:42:43.302909 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:43.302921 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:43.302960 (+ 39us) server_negotiation.cc:407] Connection header received
0807 19:42:43.305378 (+ 2418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:43.305401 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:43.305806 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:43.306232 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:43.306802 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:43.308196 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:43.309552 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:43.310250 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:43.310460 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:43.311844 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:43.311858 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:43.311878 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:43.312441 (+ 563us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:43.312515 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:43.312539 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:43.312718 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:43.312984 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 19:42:43.313157 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":344,"threads_started":1}
I20260807 19:42:43.317241 3196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:43.318292 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:43.318585 3196 log.cc:824] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:43.321316 3196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75: No bootstrap required, opened a new log
I20260807 19:42:43.330178 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:43.322827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57835 (local address 127.0.0.1:33555)
0807 19:42:43.323037 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:43.323043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:43.323676 (+ 633us) server_negotiation.cc:407] Connection header received
0807 19:42:43.323887 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:43.323892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:43.323963 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:43.324059 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:43.324697 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:43.325536 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:43.327521 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:43.328000 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:43.328105 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:43.329355 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:43.329363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:43.329367 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:43.329383 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:43.329418 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:43.329433 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:43.329524 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:43.329740 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 19:42:43.329811 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:42:43.331614 3196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3d99f73f794920ad231e73f8c75d75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33555 } }
I20260807 19:42:43.332132 3196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:43.332254 3196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f3d99f73f794920ad231e73f8c75d75, State: Initialized, Role: FOLLOWER
I20260807 19:42:43.332876 3196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3d99f73f794920ad231e73f8c75d75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33555 } }
I20260807 19:42:43.333117 3196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:43.333284 3196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:43.333459 3196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:43.335927 3196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3d99f73f794920ad231e73f8c75d75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33555 } }
I20260807 19:42:43.336469 3196 leader_election.cc:302] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f3d99f73f794920ad231e73f8c75d75; no voters:
I20260807 19:42:43.337095 3196 leader_election.cc:288] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:43.337226 3206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:43.339067 3206 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [term 1 LEADER]: Becoming Leader. State: Replica: 7f3d99f73f794920ad231e73f8c75d75, State: Running, Role: LEADER
I20260807 19:42:43.339265 3196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:43.339684 3206 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3d99f73f794920ad231e73f8c75d75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33555 } }
I20260807 19:42:43.342653 3208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f3d99f73f794920ad231e73f8c75d75. Latest consensus state: current_term: 1 leader_uuid: "7f3d99f73f794920ad231e73f8c75d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3d99f73f794920ad231e73f8c75d75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33555 } } }
I20260807 19:42:43.343115 3208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:43.342998 3207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f3d99f73f794920ad231e73f8c75d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f3d99f73f794920ad231e73f8c75d75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33555 } } }
I20260807 19:42:43.343425 3207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:43.344133 3215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:43.350129 3215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:43.362653 3215 catalog_manager.cc:1347] Generated new cluster ID: 75662f456597432f83749e697502bf6c
I20260807 19:42:43.362790 3215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:43.378328 3215 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:43.379851 3215 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:43.396327 3215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f3d99f73f794920ad231e73f8c75d75: Generated new TSK 0
I20260807 19:42:43.397207 3215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:43.408366 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3127
W20260807 19:42:43.409674 3191 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36769
W20260807 19:42:43.410426 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3127: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:42:43.410635 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 3127
W20260807 19:42:43.430936 3112 connection.cc:535] client connection to 127.0.0.1:33555 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33555): Connection reset by peer (error 104)
2026-08-07T19:42:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2062 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:43 dist-test-slave-07c2 krb5kdc[3227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:43 dist-test-slave-07c2 krb5kdc[3227](info): set up 2 sockets
Aug 07 19:42:43 dist-test-slave-07c2 krb5kdc[3227](info): commencing operation
krb5kdc: starting...
W20260807 19:42:44.560103 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:44 dist-test-slave-07c2 krb5kdc[3227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:42:44.876058 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42189
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:45.215247 3240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:45.215626 3240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:45.215814 3240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:45.222977 3240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:45.223134 3240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:45.223248 3240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:45.223372 3240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:45.223453 3240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:45.233101 3240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42189
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:45.234437 3240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:45.236018 3240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:45.248445 3248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:45.248479 3246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:45.248747 3245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:45.249643 3240 server_base.cc:1034] running on GCE node
I20260807 19:42:45.250427 3240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:45.251792 3240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:45.253003 3240 hybrid_clock.cc:648] HybridClock initialized: now 1786131765252941 us; error 48 us; skew 500 ppm
I20260807 19:42:45.253319 3240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:45 dist-test-slave-07c2 krb5kdc[3227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131765, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:45.261940 3240 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:42:45.263531 3240 webserver.cc:460] Webserver started at http://127.0.0.1:37523/ using document root <none> and password file <none>
I20260807 19:42:45.264200 3240 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:45.264340 3240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:45.264650 3240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:45.267686 3240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "0af99525a7b048218c05e5d918a75673"
format_stamp: "Formatted at 2026-08-07 19:42:45 on dist-test-slave-07c2"
server_key: "94e006f56a2ceab737dd5d2ed48c5f9a"
server_key_iv: "66890bc341c0d8ce1fe845c33b7539eb"
server_key_version: "encryptionkey@0"
I20260807 19:42:45.268463 3240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "0af99525a7b048218c05e5d918a75673"
format_stamp: "Formatted at 2026-08-07 19:42:45 on dist-test-slave-07c2"
server_key: "94e006f56a2ceab737dd5d2ed48c5f9a"
server_key_iv: "66890bc341c0d8ce1fe845c33b7539eb"
server_key_version: "encryptionkey@0"
I20260807 19:42:45.273530 3240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:42:45.276968 3255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:45.278201 3240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:42:45.278370 3240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "0af99525a7b048218c05e5d918a75673"
format_stamp: "Formatted at 2026-08-07 19:42:45 on dist-test-slave-07c2"
server_key: "94e006f56a2ceab737dd5d2ed48c5f9a"
server_key_iv: "66890bc341c0d8ce1fe845c33b7539eb"
server_key_version: "encryptionkey@0"
I20260807 19:42:45.278561 3240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:45.284245 3240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:45.288075 3240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:45.288378 3240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:45.307276 3240 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42189
I20260807 19:42:45.307376 3306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42189 every 8 connection(s)
I20260807 19:42:45.308707 3240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:45.312418 3307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:45.315137 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 3240
I20260807 19:42:45.315325 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:45.315863 14401 external_mini_cluster.cc:1428] Setting key beca2cdf4006c09d1df77704fea675b0
I20260807 19:42:45.323781 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:45.317700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36941 (local address 127.0.0.1:42189)
0807 19:42:45.318534 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:45.318545 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:45.318585 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:42:45.320298 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:45.320320 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:45.320632 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:45.320930 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:45.322130 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:45.322137 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:45.322148 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:45.322163 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:45.322227 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:45.322246 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:45.322421 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:45.322575 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 19:42:45.322800 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":412,"threads_started":1}
I20260807 19:42:45.325418 3307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673: Bootstrap starting.
I20260807 19:42:45.327206 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:45.329394 3307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:45.330845 3307 log.cc:824] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:45.334015 3307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673: No bootstrap required, opened a new log
I20260807 19:42:45.335875 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:45.333323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59785 (local address 127.0.0.1:42189)
0807 19:42:45.333500 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:45.333505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:45.333877 (+ 372us) server_negotiation.cc:407] Connection header received
0807 19:42:45.334034 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:45.334041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:45.334101 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:45.334204 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:45.335173 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:45.335179 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:45.335183 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:45.335201 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:45.335265 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:45.335312 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:45.335430 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:45.335531 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 19:42:45.335593 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:42:45.341894 3307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af99525a7b048218c05e5d918a75673" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42189 } }
I20260807 19:42:45.342257 3307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:45.342371 3307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af99525a7b048218c05e5d918a75673, State: Initialized, Role: FOLLOWER
I20260807 19:42:45.342967 3307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af99525a7b048218c05e5d918a75673" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42189 } }
I20260807 19:42:45.343197 3307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:45.343370 3307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:45.343536 3307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:45.345690 3307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af99525a7b048218c05e5d918a75673" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42189 } }
I20260807 19:42:45.346160 3307 leader_election.cc:302] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0af99525a7b048218c05e5d918a75673; no voters:
I20260807 19:42:45.346729 3307 leader_election.cc:288] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:45.346871 3317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:45.347572 3317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [term 1 LEADER]: Becoming Leader. State: Replica: 0af99525a7b048218c05e5d918a75673, State: Running, Role: LEADER
I20260807 19:42:45.348135 3317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af99525a7b048218c05e5d918a75673" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42189 } }
I20260807 19:42:45.348824 3307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:45.351238 3318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0af99525a7b048218c05e5d918a75673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af99525a7b048218c05e5d918a75673" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42189 } } }
I20260807 19:42:45.351698 3318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:45.351223 3319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0af99525a7b048218c05e5d918a75673. Latest consensus state: current_term: 1 leader_uuid: "0af99525a7b048218c05e5d918a75673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af99525a7b048218c05e5d918a75673" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42189 } } }
I20260807 19:42:45.352106 3319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:45.356379 3324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:45.361588 3324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:45.371513 3324 catalog_manager.cc:1347] Generated new cluster ID: c298762357604faab913e82303f65264
I20260807 19:42:45.371639 3324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:45.382704 3324 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:45.383163 3324 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:45.391374 3324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0af99525a7b048218c05e5d918a75673: Generated new TSK 0
I20260807 19:42:45.392303 3324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:45.407547 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3240
W20260807 19:42:45.408759 3302 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36941
W20260807 19:42:45.409551 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3240: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:42:45.409714 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 3240
2026-08-07T19:42:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1998 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:45 dist-test-slave-07c2 krb5kdc[3338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:45 dist-test-slave-07c2 krb5kdc[3338](info): set up 2 sockets
Aug 07 19:42:45 dist-test-slave-07c2 krb5kdc[3338](info): commencing operation
krb5kdc: starting...
W20260807 19:42:46.564433 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.005s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:46 dist-test-slave-07c2 krb5kdc[3338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:42:46.888509 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43339
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:47.227800 3352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:47.228134 3352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:47.228307 3352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:47.235579 3352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:47.235733 3352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:47.235845 3352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:47.235962 3352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:47.236040 3352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:47.247397 3352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:47.248788 3352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:47.250041 3352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:47.261955 3358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:47.262020 3360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:47.262017 3357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:47.263197 3352 server_base.cc:1034] running on GCE node
I20260807 19:42:47.263991 3352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:47.265367 3352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:47.266563 3352 hybrid_clock.cc:648] HybridClock initialized: now 1786131767266500 us; error 53 us; skew 500 ppm
I20260807 19:42:47.266871 3352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:47 dist-test-slave-07c2 krb5kdc[3338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131767, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:47.270027 3352 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:42:47.271605 3352 webserver.cc:460] Webserver started at http://127.0.0.1:45877/ using document root <none> and password file <none>
I20260807 19:42:47.272239 3352 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:47.272351 3352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:47.272600 3352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:47.275611 3352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "ea8ebc6f701b4da991fa72b2858d4b45"
format_stamp: "Formatted at 2026-08-07 19:42:47 on dist-test-slave-07c2"
server_key: "41ada5435e892f2173078176db6b0a50"
server_key_iv: "f5d986a8befed6d4d5d9fe1f35471313"
server_key_version: "encryptionkey@0"
I20260807 19:42:47.276345 3352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "ea8ebc6f701b4da991fa72b2858d4b45"
format_stamp: "Formatted at 2026-08-07 19:42:47 on dist-test-slave-07c2"
server_key: "41ada5435e892f2173078176db6b0a50"
server_key_iv: "f5d986a8befed6d4d5d9fe1f35471313"
server_key_version: "encryptionkey@0"
I20260807 19:42:47.281361 3352 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:42:47.284634 3368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:47.285908 3352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 19:42:47.286057 3352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "ea8ebc6f701b4da991fa72b2858d4b45"
format_stamp: "Formatted at 2026-08-07 19:42:47 on dist-test-slave-07c2"
server_key: "41ada5435e892f2173078176db6b0a50"
server_key_iv: "f5d986a8befed6d4d5d9fe1f35471313"
server_key_version: "encryptionkey@0"
I20260807 19:42:47.286211 3352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:47.317618 3352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:47.321666 3352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:47.321908 3352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:47.340919 3352 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43339
I20260807 19:42:47.340955 3419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43339 every 8 connection(s)
I20260807 19:42:47.342320 3352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:47.346061 3420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:47.349828 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 3352
I20260807 19:42:47.350014 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:47.350459 14401 external_mini_cluster.cc:1428] Setting key 6b878f6974a3050b592dab5cf141207a
I20260807 19:42:47.358654 3420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45: Bootstrap starting.
I20260807 19:42:47.362360 3420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:47.363729 3420 log.cc:824] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:42:47 dist-test-slave-07c2 krb5kdc[3338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:42:47.366442 3420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45: No bootstrap required, opened a new log
I20260807 19:42:47.373361 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.352179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38181 (local address 127.0.0.1:43339)
0807 19:42:47.352945 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.352956 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.352996 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:42:47.354577 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.354598 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.354976 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.355339 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:47.357058 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.358330 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.360138 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.360792 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.360913 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.366009 (+ 5096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:47.366017 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:47.366030 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:47.366063 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:47.369361 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:47.370070 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:47.370076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:47.370082 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:47.370262 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:47.370650 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:47.370657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:47.370659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:47.371191 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:47.371530 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.371988 (+ 458us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.372400 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":361,"threads_started":1}
I20260807 19:42:47.374619 3420 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8ebc6f701b4da991fa72b2858d4b45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43339 } }
I20260807 19:42:47.375072 3420 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:47.375190 3420 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea8ebc6f701b4da991fa72b2858d4b45, State: Initialized, Role: FOLLOWER
I20260807 19:42:47.375818 3420 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8ebc6f701b4da991fa72b2858d4b45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43339 } }
I20260807 19:42:47.376094 3420 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:47.376255 3420 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:47.376458 3420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:47.377266 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:47.378845 3420 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8ebc6f701b4da991fa72b2858d4b45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43339 } }
I20260807 19:42:47.379415 3420 leader_election.cc:302] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea8ebc6f701b4da991fa72b2858d4b45; no voters:
I20260807 19:42:47.380105 3420 leader_election.cc:288] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:47.380218 3426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:47.382087 3426 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [term 1 LEADER]: Becoming Leader. State: Replica: ea8ebc6f701b4da991fa72b2858d4b45, State: Running, Role: LEADER
I20260807 19:42:47.382776 3420 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:47.382743 3426 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8ebc6f701b4da991fa72b2858d4b45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43339 } }
I20260807 19:42:47.393586 3429 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea8ebc6f701b4da991fa72b2858d4b45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8ebc6f701b4da991fa72b2858d4b45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43339 } } }
I20260807 19:42:47.394062 3429 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:47.394121 3431 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea8ebc6f701b4da991fa72b2858d4b45. Latest consensus state: current_term: 1 leader_uuid: "ea8ebc6f701b4da991fa72b2858d4b45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8ebc6f701b4da991fa72b2858d4b45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43339 } } }
I20260807 19:42:47.394428 3431 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:47.402565 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.383494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35073 (local address 127.0.0.1:43339)
0807 19:42:47.383877 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.383883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.383910 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:42:47.383953 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.383956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.384008 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.384135 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:47.385835 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.386593 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.387954 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.388397 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.388530 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.391570 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:47.391578 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:47.391584 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:47.391631 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:47.397379 (+ 5748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:47.398513 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:47.398522 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:47.398529 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:47.398772 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:47.399365 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:47.399373 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:47.399379 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:47.399894 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:47.400210 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.401746 (+ 1536us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.402046 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259}
I20260807 19:42:47.407586 3438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:47.413254 3438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:47.421712 3438 catalog_manager.cc:1347] Generated new cluster ID: 81837e7d024644e4822e537b48c8ac92
I20260807 19:42:47.421849 3438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:47.440744 3438 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:47.442584 3438 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:47.458968 3438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea8ebc6f701b4da991fa72b2858d4b45: Generated new TSK 0
I20260807 19:42:47.459932 3438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:47.476890 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3352
I20260807 19:42:47.478114 3415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:47.571679 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 3352
2026-08-07T19:42:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2164 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:47 dist-test-slave-07c2 krb5kdc[3453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:47 dist-test-slave-07c2 krb5kdc[3453](info): set up 2 sockets
Aug 07 19:42:47 dist-test-slave-07c2 krb5kdc[3453](info): commencing operation
krb5kdc: starting...
W20260807 19:42:48.718844 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:48 dist-test-slave-07c2 krb5kdc[3453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:42:49.037925 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36945
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:49.376502 3466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:49.376813 3466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:49.376909 3466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:49.384243 3466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:49.384371 3466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:49.384419 3466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:49.384482 3466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:49.384523 3466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:49.394460 3466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:49.395833 3466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:49.397130 3466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:49.409356 3474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:49.409412 3472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:49.409415 3471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:49.410204 3466 server_base.cc:1034] running on GCE node
I20260807 19:42:49.411041 3466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:49.412508 3466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:49.413739 3466 hybrid_clock.cc:648] HybridClock initialized: now 1786131769413693 us; error 61 us; skew 500 ppm
I20260807 19:42:49.414126 3466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:49 dist-test-slave-07c2 krb5kdc[3453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131769, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:49.417354 3466 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:42:49.418886 3466 webserver.cc:460] Webserver started at http://127.0.0.1:37443/ using document root <none> and password file <none>
I20260807 19:42:49.419610 3466 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:49.419721 3466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:49.420042 3466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:49.423025 3466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "b6a586ca4dea47d8bf831c7d7f75a5b9"
format_stamp: "Formatted at 2026-08-07 19:42:49 on dist-test-slave-07c2"
server_key: "fe46d8b40a57a3d10700d56d0e95b92f"
server_key_iv: "f985489b7391797c8864736a3ef7b83f"
server_key_version: "encryptionkey@0"
I20260807 19:42:49.423830 3466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "b6a586ca4dea47d8bf831c7d7f75a5b9"
format_stamp: "Formatted at 2026-08-07 19:42:49 on dist-test-slave-07c2"
server_key: "fe46d8b40a57a3d10700d56d0e95b92f"
server_key_iv: "f985489b7391797c8864736a3ef7b83f"
server_key_version: "encryptionkey@0"
I20260807 19:42:49.428977 3466 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:42:49.432410 3481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:49.433691 3466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:49.433863 3466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "b6a586ca4dea47d8bf831c7d7f75a5b9"
format_stamp: "Formatted at 2026-08-07 19:42:49 on dist-test-slave-07c2"
server_key: "fe46d8b40a57a3d10700d56d0e95b92f"
server_key_iv: "f985489b7391797c8864736a3ef7b83f"
server_key_version: "encryptionkey@0"
I20260807 19:42:49.434058 3466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:49.460528 3466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:49.464995 3466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:49.465314 3466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:49.484963 3466 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36945
I20260807 19:42:49.484988 3532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36945 every 8 connection(s)
I20260807 19:42:49.486430 3466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:49.488507 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 3466
I20260807 19:42:49.488720 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:49.489316 14401 external_mini_cluster.cc:1428] Setting key d46cf29e207d89fb2d2aff4724bf9305
I20260807 19:42:49.491248 3533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:49.502792 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:49.491130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55987 (local address 127.0.0.1:36945)
0807 19:42:49.492074 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:49.492085 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:49.492129 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:42:49.493713 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:49.493734 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:49.494124 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:49.494449 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:49.495064 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:49.496516 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:49.498366 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:49.499003 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:49.499129 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:49.500466 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:49.500473 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:49.500490 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:49.501078 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:49.501151 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:49.501181 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:49.501281 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:49.501553 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 19:42:49.501888 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":505,"threads_started":1}
I20260807 19:42:49.504112 3533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9: Bootstrap starting.
I20260807 19:42:49.506431 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:49.507434 3533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:49.508661 3533 log.cc:824] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:49.511503 3533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9: No bootstrap required, opened a new log
I20260807 19:42:49.518533 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:49.511803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56651 (local address 127.0.0.1:36945)
0807 19:42:49.512009 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:49.512017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:49.512397 (+ 380us) server_negotiation.cc:407] Connection header received
0807 19:42:49.512588 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:49.512593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:49.512668 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:49.512786 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:49.513290 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:49.514471 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:49.515814 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:49.516549 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:49.516677 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:49.517698 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:49.517705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:49.517710 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:49.517730 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:49.517777 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:49.517796 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:49.517889 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:49.518081 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 19:42:49.518181 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:42:49.521155 3533 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a586ca4dea47d8bf831c7d7f75a5b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36945 } }
I20260807 19:42:49.521531 3533 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:49.521647 3533 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6a586ca4dea47d8bf831c7d7f75a5b9, State: Initialized, Role: FOLLOWER
I20260807 19:42:49.522264 3533 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a586ca4dea47d8bf831c7d7f75a5b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36945 } }
I20260807 19:42:49.522487 3533 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:49.522642 3533 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:49.522823 3533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:49.525445 3533 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a586ca4dea47d8bf831c7d7f75a5b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36945 } }
I20260807 19:42:49.525998 3533 leader_election.cc:302] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6a586ca4dea47d8bf831c7d7f75a5b9; no voters:
I20260807 19:42:49.526652 3533 leader_election.cc:288] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:49.526881 3544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:49.528244 3544 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [term 1 LEADER]: Becoming Leader. State: Replica: b6a586ca4dea47d8bf831c7d7f75a5b9, State: Running, Role: LEADER
I20260807 19:42:49.528885 3544 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a586ca4dea47d8bf831c7d7f75a5b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36945 } }
I20260807 19:42:49.529405 3533 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:49.532109 3545 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6a586ca4dea47d8bf831c7d7f75a5b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a586ca4dea47d8bf831c7d7f75a5b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36945 } } }
I20260807 19:42:49.532168 3546 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6a586ca4dea47d8bf831c7d7f75a5b9. Latest consensus state: current_term: 1 leader_uuid: "b6a586ca4dea47d8bf831c7d7f75a5b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a586ca4dea47d8bf831c7d7f75a5b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36945 } } }
I20260807 19:42:49.532568 3545 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:49.532578 3546 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:49.535158 3550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:49.541954 3550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:49.551859 3550 catalog_manager.cc:1347] Generated new cluster ID: a9348b511cf34898bb05e68165b20ddb
I20260807 19:42:49.551999 3550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:49.562192 3550 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:49.563773 3550 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:49.572283 3550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6a586ca4dea47d8bf831c7d7f75a5b9: Generated new TSK 0
I20260807 19:42:49.573385 3550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:49.580799 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3466
W20260807 19:42:49.582036 3528 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55987
W20260807 19:42:49.582958 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3466: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:42:49.583114 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 3466
2026-08-07T19:42:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2009 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:49 dist-test-slave-07c2 krb5kdc[3566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:49 dist-test-slave-07c2 krb5kdc[3566](info): set up 2 sockets
Aug 07 19:42:49 dist-test-slave-07c2 krb5kdc[3566](info): commencing operation
krb5kdc: starting...
W20260807 19:42:50.735271 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.008s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:50 dist-test-slave-07c2 krb5kdc[3566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:42:51.057940 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40549
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:51.404837 3579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:51.405150 3579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:51.405337 3579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:51.412798 3579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:51.412933 3579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:51.413065 3579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:51.413172 3579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:51.413259 3579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:51.423990 3579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:51.425285 3579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:51.426595 3579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:51.439028 3584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:51.439003 3585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:51.439616 3587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:51.440177 3579 server_base.cc:1034] running on GCE node
I20260807 19:42:51.441306 3579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:51.443236 3579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:51.444564 3579 hybrid_clock.cc:648] HybridClock initialized: now 1786131771444435 us; error 90 us; skew 500 ppm
I20260807 19:42:51.445034 3579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:51 dist-test-slave-07c2 krb5kdc[3566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:51.449306 3579 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:42:51.451407 3579 webserver.cc:460] Webserver started at http://127.0.0.1:41975/ using document root <none> and password file <none>
I20260807 19:42:51.452421 3579 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:51.452581 3579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:51.452976 3579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:51.457283 3579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "b9690e91c8484ae7869dc5e9adefdaeb"
format_stamp: "Formatted at 2026-08-07 19:42:51 on dist-test-slave-07c2"
server_key: "45ff61fb647b9ca4f3b914b90cd5ee3d"
server_key_iv: "6105c6f65fbee1ce3871e2d517035b72"
server_key_version: "encryptionkey@0"
I20260807 19:42:51.458292 3579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "b9690e91c8484ae7869dc5e9adefdaeb"
format_stamp: "Formatted at 2026-08-07 19:42:51 on dist-test-slave-07c2"
server_key: "45ff61fb647b9ca4f3b914b90cd5ee3d"
server_key_iv: "6105c6f65fbee1ce3871e2d517035b72"
server_key_version: "encryptionkey@0"
I20260807 19:42:51.465865 3579 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260807 19:42:51.471132 3594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:51.472981 3579 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:51.473181 3579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "b9690e91c8484ae7869dc5e9adefdaeb"
format_stamp: "Formatted at 2026-08-07 19:42:51 on dist-test-slave-07c2"
server_key: "45ff61fb647b9ca4f3b914b90cd5ee3d"
server_key_iv: "6105c6f65fbee1ce3871e2d517035b72"
server_key_version: "encryptionkey@0"
I20260807 19:42:51.473443 3579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:51.482190 3579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:51.488209 3579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:51.488562 3579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:51.507929 3579 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40549
I20260807 19:42:51.508013 3645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40549 every 8 connection(s)
I20260807 19:42:51.509326 3579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:51.510833 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 3579
I20260807 19:42:51.511008 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:51.511497 14401 external_mini_cluster.cc:1428] Setting key 6fd54bd14e51b68ed9933e9326ffc417
I20260807 19:42:51.513525 3646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:51.519099 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:51.513320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40763 (local address 127.0.0.1:40549)
0807 19:42:51.514152 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:51.514162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:51.514199 (+ 37us) server_negotiation.cc:407] Connection header received
0807 19:42:51.515812 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:51.515832 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:51.516160 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:51.516467 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:51.517559 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:51.517565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:51.517576 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:51.517596 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:51.517663 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:51.517683 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:51.517867 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:51.517967 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 19:42:51.518153 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":374,"threads_started":1}
I20260807 19:42:51.522629 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:51.526314 3646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb: Bootstrap starting.
I20260807 19:42:51.530277 3646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:51.530666 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:51.528233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35477 (local address 127.0.0.1:40549)
0807 19:42:51.528451 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:51.528460 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:51.528683 (+ 223us) server_negotiation.cc:407] Connection header received
0807 19:42:51.528967 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:51.528971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:51.529017 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:51.529121 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:51.530026 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:51.530031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:51.530036 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:51.530054 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:51.530099 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:51.530114 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:51.530236 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:51.530335 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 19:42:51.530395 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:42:51.531646 3646 log.cc:824] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:51.534709 3646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb: No bootstrap required, opened a new log
I20260807 19:42:51.542589 3646 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9690e91c8484ae7869dc5e9adefdaeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40549 } }
I20260807 19:42:51.543022 3646 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:51.543143 3646 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9690e91c8484ae7869dc5e9adefdaeb, State: Initialized, Role: FOLLOWER
I20260807 19:42:51.543848 3646 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9690e91c8484ae7869dc5e9adefdaeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40549 } }
I20260807 19:42:51.544087 3646 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:51.544245 3646 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:51.544442 3646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:51.547041 3646 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9690e91c8484ae7869dc5e9adefdaeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40549 } }
I20260807 19:42:51.547606 3646 leader_election.cc:302] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9690e91c8484ae7869dc5e9adefdaeb; no voters:
I20260807 19:42:51.548188 3646 leader_election.cc:288] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:51.548336 3657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:51.549045 3657 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [term 1 LEADER]: Becoming Leader. State: Replica: b9690e91c8484ae7869dc5e9adefdaeb, State: Running, Role: LEADER
I20260807 19:42:51.549551 3657 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9690e91c8484ae7869dc5e9adefdaeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40549 } }
I20260807 19:42:51.550316 3646 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:51.552755 3658 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9690e91c8484ae7869dc5e9adefdaeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9690e91c8484ae7869dc5e9adefdaeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40549 } } }
I20260807 19:42:51.553182 3658 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:51.553004 3659 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9690e91c8484ae7869dc5e9adefdaeb. Latest consensus state: current_term: 1 leader_uuid: "b9690e91c8484ae7869dc5e9adefdaeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9690e91c8484ae7869dc5e9adefdaeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40549 } } }
I20260807 19:42:51.553452 3659 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:51.555847 3664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:51.561062 3664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:51.569787 3664 catalog_manager.cc:1347] Generated new cluster ID: 69e89a257d0a4939bb55360ff1bb02ba
I20260807 19:42:51.569906 3664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:51.621752 3664 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:51.622289 3664 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:51.630934 3664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9690e91c8484ae7869dc5e9adefdaeb: Generated new TSK 0
I20260807 19:42:51.631870 3664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:51.642418 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3579
W20260807 19:42:51.643901 3641 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40763
W20260807 19:42:51.644640 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3579: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:42:51.644805 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 3579
2026-08-07T19:42:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2060 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:51 dist-test-slave-07c2 krb5kdc[3679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:51 dist-test-slave-07c2 krb5kdc[3679](info): set up 2 sockets
Aug 07 19:42:51 dist-test-slave-07c2 krb5kdc[3679](info): commencing operation
krb5kdc: starting...
W20260807 19:42:52.808833 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:53 dist-test-slave-07c2 krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:42:53.130445 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37503
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:53.478240 3692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:53.478551 3692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:53.478765 3692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:53.486179 3692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:53.486292 3692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:53.486429 3692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:53.486533 3692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:53.486620 3692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:53.496884 3692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:53.498240 3692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:53.499627 3692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:53.511404 3700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:53.511508 3698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:53.511698 3697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:53.512401 3692 server_base.cc:1034] running on GCE node
I20260807 19:42:53.513145 3692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:53.514597 3692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:53.515825 3692 hybrid_clock.cc:648] HybridClock initialized: now 1786131773515774 us; error 38 us; skew 500 ppm
I20260807 19:42:53.516203 3692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:53 dist-test-slave-07c2 krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:53.519459 3692 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:42:53.520931 3692 webserver.cc:460] Webserver started at http://127.0.0.1:38283/ using document root <none> and password file <none>
I20260807 19:42:53.521552 3692 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:53.521662 3692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:53.521910 3692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:53.524857 3692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "e140c1fcb7404d4881bf954434f90cf3"
format_stamp: "Formatted at 2026-08-07 19:42:53 on dist-test-slave-07c2"
server_key: "c27548ac3a2d72c4c88627c75f0965a3"
server_key_iv: "421e8b0698964984c6c8e3c48858647d"
server_key_version: "encryptionkey@0"
I20260807 19:42:53.525647 3692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "e140c1fcb7404d4881bf954434f90cf3"
format_stamp: "Formatted at 2026-08-07 19:42:53 on dist-test-slave-07c2"
server_key: "c27548ac3a2d72c4c88627c75f0965a3"
server_key_iv: "421e8b0698964984c6c8e3c48858647d"
server_key_version: "encryptionkey@0"
I20260807 19:42:53.530678 3692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:42:53.534173 3707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:53.535544 3692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:53.535732 3692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "e140c1fcb7404d4881bf954434f90cf3"
format_stamp: "Formatted at 2026-08-07 19:42:53 on dist-test-slave-07c2"
server_key: "c27548ac3a2d72c4c88627c75f0965a3"
server_key_iv: "421e8b0698964984c6c8e3c48858647d"
server_key_version: "encryptionkey@0"
I20260807 19:42:53.535923 3692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:53.552083 3692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:53.556617 3692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:53.556922 3692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:53.576586 3759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37503 every 8 connection(s)
I20260807 19:42:53.576620 3692 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37503
I20260807 19:42:53.578711 3692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:53.583613 3760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:53.583647 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 3692
I20260807 19:42:53.583900 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:53.584347 14401 external_mini_cluster.cc:1428] Setting key e85f6286100758eee2ac0ded75234f89
I20260807 19:42:53.598011 3760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3: Bootstrap starting.
Aug 07 19:42:53 dist-test-slave-07c2 krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:42:53.601614 3760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:53.603024 3760 log.cc:824] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:53.606392 3760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3: No bootstrap required, opened a new log
I20260807 19:42:53.608969 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:53.586210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:52167 (local address 127.0.0.1:37503)
0807 19:42:53.587070 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:53.587086 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:53.587134 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:42:53.588886 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:53.588920 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:53.589452 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:53.589906 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:53.591217 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:53.593180 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:53.594485 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:53.595615 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:53.595825 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:53.600123 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:53.600134 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:53.600151 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:53.600200 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:53.604232 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:53.605090 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:53.605098 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:53.605106 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:53.605320 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:53.605771 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:53.605775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:53.605778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:53.606481 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:53.606804 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:53.607260 (+ 456us) server_negotiation.cc:299] Negotiation successful
0807 19:42:53.607609 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":319,"threads_started":1}
I20260807 19:42:53.612874 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:53.615602 3760 raft_consensus.cc:348] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e140c1fcb7404d4881bf954434f90cf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260807 19:42:53.616029 3760 raft_consensus.cc:374] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:53.616151 3760 raft_consensus.cc:729] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e140c1fcb7404d4881bf954434f90cf3, State: Initialized, Role: FOLLOWER
I20260807 19:42:53.616716 3760 consensus_queue.cc:260] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e140c1fcb7404d4881bf954434f90cf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260807 19:42:53.616972 3760 raft_consensus.cc:388] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:53.617128 3760 raft_consensus.cc:482] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:53.617398 3760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:53.619895 3760 raft_consensus.cc:504] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e140c1fcb7404d4881bf954434f90cf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260807 19:42:53.620414 3760 leader_election.cc:302] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e140c1fcb7404d4881bf954434f90cf3; no voters:
I20260807 19:42:53.621094 3760 leader_election.cc:288] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:53.621204 3770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:53.623778 3760 sys_catalog.cc:564] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:53.628027 3770 raft_consensus.cc:686] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [term 1 LEADER]: Becoming Leader. State: Replica: e140c1fcb7404d4881bf954434f90cf3, State: Running, Role: LEADER
I20260807 19:42:53.628921 3770 consensus_queue.cc:237] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e140c1fcb7404d4881bf954434f90cf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } }
I20260807 19:42:53.632670 3771 sys_catalog.cc:455] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e140c1fcb7404d4881bf954434f90cf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e140c1fcb7404d4881bf954434f90cf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } } }
I20260807 19:42:53.633059 3771 sys_catalog.cc:458] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:53.634040 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:53.617399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:51841 (local address 127.0.0.1:37503)
0807 19:42:53.617571 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:53.617579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:53.618065 (+ 486us) server_negotiation.cc:407] Connection header received
0807 19:42:53.618269 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:53.618275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:53.618355 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:53.618477 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:53.620025 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:53.620931 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:53.622648 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:53.623169 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:53.623331 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:53.626056 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:53.626061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:53.626065 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:53.626096 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:53.629712 (+ 3616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:53.630488 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:53.630494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:53.630498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:53.630642 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:53.631103 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:53.631107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:53.631111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:53.631625 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:53.633089 (+ 1464us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:53.633493 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 19:42:53.633669 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 19:42:53.638978 3772 sys_catalog.cc:455] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e140c1fcb7404d4881bf954434f90cf3. Latest consensus state: current_term: 1 leader_uuid: "e140c1fcb7404d4881bf954434f90cf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e140c1fcb7404d4881bf954434f90cf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37503 } } }
I20260807 19:42:53.639420 3772 sys_catalog.cc:458] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:53.644263 3779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:53.650429 3779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:53.659106 3779 catalog_manager.cc:1347] Generated new cluster ID: d6969573c8444afa94980bfb2c95720b
I20260807 19:42:53.659217 3779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:53.733052 3779 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:53.734982 3779 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:53.747990 3779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e140c1fcb7404d4881bf954434f90cf3: Generated new TSK 0
I20260807 19:42:53.748842 3779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:53.755973 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3692
I20260807 19:42:53.757102 3755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:53.853348 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 3692
W20260807 19:42:53.873800 3677 connection.cc:535] client connection to 127.0.0.1:37503 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37503): Connection reset by peer (error 104)
2026-08-07T19:42:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2209 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:53 dist-test-slave-07c2 krb5kdc[3792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:53 dist-test-slave-07c2 krb5kdc[3792](info): set up 2 sockets
Aug 07 19:42:53 dist-test-slave-07c2 krb5kdc[3792](info): commencing operation
krb5kdc: starting...
W20260807 19:42:55.006594 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.005s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:55 dist-test-slave-07c2 krb5kdc[3792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:42:55.334455 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46031
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:55.675952 3806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:55.676229 3806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:55.676432 3806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:55.683526 3806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:55.683663 3806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:55.683775 3806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:55.683881 3806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:55.683951 3806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:55.694202 3806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:55.695478 3806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:55.696749 3806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:55.708581 3811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:55.708600 3812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:55.708495 3815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:55.709360 3806 server_base.cc:1034] running on GCE node
I20260807 19:42:55.710103 3806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:55.711515 3806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:55.712747 3806 hybrid_clock.cc:648] HybridClock initialized: now 1786131775712710 us; error 40 us; skew 500 ppm
I20260807 19:42:55.713086 3806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:55 dist-test-slave-07c2 krb5kdc[3792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:55.716284 3806 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:42:55.717743 3806 webserver.cc:460] Webserver started at http://127.0.0.1:44699/ using document root <none> and password file <none>
I20260807 19:42:55.718353 3806 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:55.718456 3806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:55.718693 3806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:55.721616 3806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "628ff2306e664353b41a2e63af25d927"
format_stamp: "Formatted at 2026-08-07 19:42:55 on dist-test-slave-07c2"
server_key: "f5c1b9ec7c65dc90e7380853519745d3"
server_key_iv: "8618d6bd19b02437db549eef187260d3"
server_key_version: "encryptionkey@0"
I20260807 19:42:55.722314 3806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "628ff2306e664353b41a2e63af25d927"
format_stamp: "Formatted at 2026-08-07 19:42:55 on dist-test-slave-07c2"
server_key: "f5c1b9ec7c65dc90e7380853519745d3"
server_key_iv: "8618d6bd19b02437db549eef187260d3"
server_key_version: "encryptionkey@0"
I20260807 19:42:55.727396 3806 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 19:42:55.730579 3823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:55.732007 3806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:55.732157 3806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "628ff2306e664353b41a2e63af25d927"
format_stamp: "Formatted at 2026-08-07 19:42:55 on dist-test-slave-07c2"
server_key: "f5c1b9ec7c65dc90e7380853519745d3"
server_key_iv: "8618d6bd19b02437db549eef187260d3"
server_key_version: "encryptionkey@0"
I20260807 19:42:55.732321 3806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:55.754134 3806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:55.758106 3806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:55.758347 3806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:55.777040 3806 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46031
I20260807 19:42:55.777091 3876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46031 every 8 connection(s)
I20260807 19:42:55.778391 3806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:55.782038 3877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:55.786206 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 3806
I20260807 19:42:55.786382 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:55.786931 14401 external_mini_cluster.cc:1428] Setting key dfeb93c6564ff6bacd1222797bbd6ff9
I20260807 19:42:55.793521 3877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927: Bootstrap starting.
I20260807 19:42:55.796535 3877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:55.797554 3877 log.cc:824] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:55.800976 3877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927: No bootstrap required, opened a new log
I20260807 19:42:55.801229 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:55.788799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:60705 (local address 127.0.0.1:46031)
0807 19:42:55.789848 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:55.789859 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:55.789899 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:42:55.792196 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:55.792218 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:55.792598 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:55.792975 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:55.793500 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:55.794852 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:55.796284 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:55.797020 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:55.797217 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:55.798581 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:55.798588 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:55.798601 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:55.799259 (+ 658us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:55.799385 (+ 126us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:55.799416 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:55.799585 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:55.799839 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 19:42:55.800012 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":346,"threads_started":1}
I20260807 19:42:55.805155 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:55.811655 3877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "628ff2306e664353b41a2e63af25d927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46031 } }
I20260807 19:42:55.812052 3877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:55.812172 3877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 628ff2306e664353b41a2e63af25d927, State: Initialized, Role: FOLLOWER
I20260807 19:42:55.812800 3877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "628ff2306e664353b41a2e63af25d927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46031 } }
I20260807 19:42:55.813052 3877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:55.813210 3877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:55.813401 3877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:55.815884 3877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "628ff2306e664353b41a2e63af25d927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46031 } }
I20260807 19:42:55.816383 3877 leader_election.cc:302] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 628ff2306e664353b41a2e63af25d927; no voters:
I20260807 19:42:55.817022 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:55.810683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:53243 (local address 127.0.0.1:46031)
0807 19:42:55.810883 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:55.810889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:55.811220 (+ 331us) server_negotiation.cc:407] Connection header received
0807 19:42:55.811410 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:55.811415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:55.811492 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:55.811590 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:55.812108 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:55.812929 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:55.814305 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:55.814810 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:55.814965 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:55.816120 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:55.816126 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:55.816135 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:55.816153 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:55.816212 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:55.816228 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:55.816344 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:55.816568 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 19:42:55.816620 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 19:42:55.817152 3877 leader_election.cc:288] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:55.817272 3887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:55.817987 3887 raft_consensus.cc:686] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [term 1 LEADER]: Becoming Leader. State: Replica: 628ff2306e664353b41a2e63af25d927, State: Running, Role: LEADER
I20260807 19:42:55.818604 3887 consensus_queue.cc:237] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "628ff2306e664353b41a2e63af25d927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46031 } }
I20260807 19:42:55.820292 3877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:55.821537 3889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "628ff2306e664353b41a2e63af25d927" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "628ff2306e664353b41a2e63af25d927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46031 } } }
I20260807 19:42:55.822005 3889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:55.823451 3888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 628ff2306e664353b41a2e63af25d927. Latest consensus state: current_term: 1 leader_uuid: "628ff2306e664353b41a2e63af25d927" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "628ff2306e664353b41a2e63af25d927" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46031 } } }
I20260807 19:42:55.823827 3888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:55.824841 3892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:55.830021 3892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:55.838232 3892 catalog_manager.cc:1347] Generated new cluster ID: a304228df6d64a17bc830a27824759a7
I20260807 19:42:55.838344 3892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:55.848132 3892 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:55.849885 3892 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:55.857754 3892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 628ff2306e664353b41a2e63af25d927: Generated new TSK 0
I20260807 19:42:55.858696 3892 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:55.873206 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3806
W20260807 19:42:55.874330 3872 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.85:60705
W20260807 19:42:55.875097 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3806: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:42:55.875241 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 3806
W20260807 19:42:55.894524 3790 connection.cc:535] client connection to 127.0.0.1:46031 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46031): Connection reset by peer (error 104)
2026-08-07T19:42:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2020 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:55 dist-test-slave-07c2 krb5kdc[3909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:55 dist-test-slave-07c2 krb5kdc[3909](info): set up 2 sockets
Aug 07 19:42:55 dist-test-slave-07c2 krb5kdc[3909](info): commencing operation
krb5kdc: starting...
W20260807 19:42:57.022691 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:57 dist-test-slave-07c2 krb5kdc[3909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:42:57.356446 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42941
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:57.702162 3922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:57.702498 3922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:57.702677 3922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:57.709883 3922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:57.710038 3922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:57.710150 3922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:57.710261 3922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:57.710340 3922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:57.720019 3922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:57.721370 3922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:57.722688 3922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:57.734936 3927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:57.734877 3928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:57.734985 3930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:57.736102 3922 server_base.cc:1034] running on GCE node
I20260807 19:42:57.736894 3922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:57.738258 3922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:57.739491 3922 hybrid_clock.cc:648] HybridClock initialized: now 1786131777739441 us; error 59 us; skew 500 ppm
I20260807 19:42:57.739845 3922 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:57 dist-test-slave-07c2 krb5kdc[3909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131777, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:57.749127 3922 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:42:57.750654 3922 webserver.cc:460] Webserver started at http://127.0.0.1:46565/ using document root <none> and password file <none>
I20260807 19:42:57.751459 3922 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:57.751616 3922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:57.751929 3922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:57.754940 3922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "eb32d25454fa40efb7c8f07f1d9b493d"
format_stamp: "Formatted at 2026-08-07 19:42:57 on dist-test-slave-07c2"
server_key: "f053a7320f8835e7f57c4a99092ee1b2"
server_key_iv: "6d6fbe9223b292777bdb51cb6f5a9648"
server_key_version: "encryptionkey@0"
I20260807 19:42:57.755762 3922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "eb32d25454fa40efb7c8f07f1d9b493d"
format_stamp: "Formatted at 2026-08-07 19:42:57 on dist-test-slave-07c2"
server_key: "f053a7320f8835e7f57c4a99092ee1b2"
server_key_iv: "6d6fbe9223b292777bdb51cb6f5a9648"
server_key_version: "encryptionkey@0"
I20260807 19:42:57.760751 3922 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:57.764122 3938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:57.765363 3922 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:57.765540 3922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "eb32d25454fa40efb7c8f07f1d9b493d"
format_stamp: "Formatted at 2026-08-07 19:42:57 on dist-test-slave-07c2"
server_key: "f053a7320f8835e7f57c4a99092ee1b2"
server_key_iv: "6d6fbe9223b292777bdb51cb6f5a9648"
server_key_version: "encryptionkey@0"
I20260807 19:42:57.765738 3922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:57.771214 3922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:57.775017 3922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:57.775384 3922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:57.793905 3922 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42941
I20260807 19:42:57.793955 3990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42941 every 8 connection(s)
I20260807 19:42:57.795331 3922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:57.796792 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 3922
I20260807 19:42:57.796967 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:57.797524 14401 external_mini_cluster.cc:1428] Setting key da798d1825a21fcddf5660b32304cb98
I20260807 19:42:57.799809 3991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:57.805110 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:57.799347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:43695 (local address 127.0.0.1:42941)
0807 19:42:57.800160 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:57.800171 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:57.800212 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:42:57.801786 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:57.801807 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:57.802125 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:57.802439 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:57.803542 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:57.803547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:57.803559 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:57.803586 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:57.803689 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:57.803710 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:57.803881 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:57.804002 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 19:42:57.804209 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":377,"threads_started":1}
I20260807 19:42:57.808673 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:57.812662 3991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d: Bootstrap starting.
I20260807 19:42:57.816397 3991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:57.816695 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:57.814128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:33479 (local address 127.0.0.1:42941)
0807 19:42:57.814363 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:57.814371 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:57.814742 (+ 371us) server_negotiation.cc:407] Connection header received
0807 19:42:57.814885 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:57.814892 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:57.814952 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:57.815063 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:57.816046 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:57.816051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:57.816055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:57.816073 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:57.816120 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:57.816144 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:57.816277 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:57.816340 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 19:42:57.816407 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:42:57.817809 3991 log.cc:824] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:57.820459 3991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d: No bootstrap required, opened a new log
I20260807 19:42:57.827800 3991 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb32d25454fa40efb7c8f07f1d9b493d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42941 } }
I20260807 19:42:57.828158 3991 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:57.828275 3991 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb32d25454fa40efb7c8f07f1d9b493d, State: Initialized, Role: FOLLOWER
I20260807 19:42:57.828828 3991 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb32d25454fa40efb7c8f07f1d9b493d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42941 } }
I20260807 19:42:57.829051 3991 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:57.829203 3991 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:57.829370 3991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:57.831512 3991 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb32d25454fa40efb7c8f07f1d9b493d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42941 } }
I20260807 19:42:57.831986 3991 leader_election.cc:302] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb32d25454fa40efb7c8f07f1d9b493d; no voters:
I20260807 19:42:57.832576 3991 leader_election.cc:288] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:57.832796 4002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:57.833496 4002 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [term 1 LEADER]: Becoming Leader. State: Replica: eb32d25454fa40efb7c8f07f1d9b493d, State: Running, Role: LEADER
I20260807 19:42:57.834115 4002 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb32d25454fa40efb7c8f07f1d9b493d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42941 } }
I20260807 19:42:57.834843 3991 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:57.837178 4004 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb32d25454fa40efb7c8f07f1d9b493d. Latest consensus state: current_term: 1 leader_uuid: "eb32d25454fa40efb7c8f07f1d9b493d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb32d25454fa40efb7c8f07f1d9b493d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42941 } } }
I20260807 19:42:57.837148 4003 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb32d25454fa40efb7c8f07f1d9b493d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb32d25454fa40efb7c8f07f1d9b493d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42941 } } }
I20260807 19:42:57.837661 4003 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:57.839144 4004 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:57.839399 4008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:57.844723 4008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:57.853597 4008 catalog_manager.cc:1347] Generated new cluster ID: c7250d9da210487ea0e23db3b96c91d8
I20260807 19:42:57.853708 4008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:57.864957 4008 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:57.865381 4008 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:57.876564 4008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb32d25454fa40efb7c8f07f1d9b493d: Generated new TSK 0
I20260807 19:42:57.877316 4008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:57.888712 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3922
W20260807 19:42:57.890336 3986 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.85:43695
W20260807 19:42:57.891160 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 3922: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:42:57.891376 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 3922
2026-08-07T19:42:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2016 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:58 dist-test-slave-07c2 krb5kdc[4023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:58 dist-test-slave-07c2 krb5kdc[4023](info): set up 2 sockets
Aug 07 19:42:58 dist-test-slave-07c2 krb5kdc[4023](info): commencing operation
krb5kdc: starting...
W20260807 19:42:59.058820 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:42:59 dist-test-slave-07c2 krb5kdc[4023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:42:59.383831 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43535
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:59.727770 4036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:59.728107 4036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:59.728287 4036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:59.735394 4036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:59.735548 4036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:59.735666 4036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:59.735795 4036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:59.735883 4036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:59.745534 4036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43535
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:42:59.746862 4036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:59.748194 4036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:59.760349 4041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:59.760378 4044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:59.760476 4042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:59.761066 4036 server_base.cc:1034] running on GCE node
I20260807 19:42:59.761853 4036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:59.763192 4036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:59.764439 4036 hybrid_clock.cc:648] HybridClock initialized: now 1786131779764396 us; error 69 us; skew 500 ppm
I20260807 19:42:59.764890 4036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:59 dist-test-slave-07c2 krb5kdc[4023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:59.767992 4036 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:42:59.769423 4036 webserver.cc:460] Webserver started at http://127.0.0.1:45269/ using document root <none> and password file <none>
I20260807 19:42:59.770093 4036 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:59.770247 4036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:59.770545 4036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:59.773538 4036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "828f459a62484b30a50b2842c50bcec2"
format_stamp: "Formatted at 2026-08-07 19:42:59 on dist-test-slave-07c2"
server_key: "7cc26998f90b9ff77a60094bafab7134"
server_key_iv: "bcde35a194881fddb9265d321efd6d62"
server_key_version: "encryptionkey@0"
I20260807 19:42:59.774327 4036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "828f459a62484b30a50b2842c50bcec2"
format_stamp: "Formatted at 2026-08-07 19:42:59 on dist-test-slave-07c2"
server_key: "7cc26998f90b9ff77a60094bafab7134"
server_key_iv: "bcde35a194881fddb9265d321efd6d62"
server_key_version: "encryptionkey@0"
I20260807 19:42:59.779451 4036 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:42:59.782793 4051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:59.784070 4036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:59.784263 4036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "828f459a62484b30a50b2842c50bcec2"
format_stamp: "Formatted at 2026-08-07 19:42:59 on dist-test-slave-07c2"
server_key: "7cc26998f90b9ff77a60094bafab7134"
server_key_iv: "bcde35a194881fddb9265d321efd6d62"
server_key_version: "encryptionkey@0"
I20260807 19:42:59.784453 4036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:59.807088 4036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:59.811096 4036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:59.811429 4036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:59.830302 4036 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43535
I20260807 19:42:59.830322 4102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43535 every 8 connection(s)
I20260807 19:42:59.831779 4036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:59.835439 4103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:59.835731 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 4036
I20260807 19:42:59.835917 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:42:59.836359 14401 external_mini_cluster.cc:1428] Setting key 56e843b2d321b5dd504a236185815b1e
I20260807 19:42:59.849306 4103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2: Bootstrap starting.
Aug 07 19:42:59 dist-test-slave-07c2 krb5kdc[4023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:42:59.852435 4103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:59.853859 4103 log.cc:824] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:59.857295 4103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2: No bootstrap required, opened a new log
I20260807 19:42:59.861086 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:59.838111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:52885 (local address 127.0.0.1:43535)
0807 19:42:59.838976 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:59.838987 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:59.839028 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:42:59.841414 (+ 2386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:59.841434 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:59.841824 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:59.842144 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:59.843942 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.845225 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.847087 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.848152 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.848344 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:59.852492 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:59.852506 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:59.852528 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:59.852573 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:59.856470 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:59.857247 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:59.857268 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:59.857283 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:59.857497 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:59.857969 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:59.857975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:59.857981 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:59.858648 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:59.858953 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:59.859398 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 19:42:59.859741 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":419,"threads_started":1}
I20260807 19:42:59.864921 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:59.866806 4103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "828f459a62484b30a50b2842c50bcec2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43535 } }
I20260807 19:42:59.867136 4103 raft_consensus.cc:374] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:59.867240 4103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 828f459a62484b30a50b2842c50bcec2, State: Initialized, Role: FOLLOWER
I20260807 19:42:59.867892 4103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "828f459a62484b30a50b2842c50bcec2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43535 } }
I20260807 19:42:59.868228 4103 raft_consensus.cc:388] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:59.868407 4103 raft_consensus.cc:482] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:59.868611 4103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:59.871381 4103 raft_consensus.cc:504] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "828f459a62484b30a50b2842c50bcec2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43535 } }
I20260807 19:42:59.871877 4103 leader_election.cc:302] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 828f459a62484b30a50b2842c50bcec2; no voters:
I20260807 19:42:59.872704 4103 leader_election.cc:288] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:59.872790 4114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:59.874286 4114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [term 1 LEADER]: Becoming Leader. State: Replica: 828f459a62484b30a50b2842c50bcec2, State: Running, Role: LEADER
I20260807 19:42:59.874965 4103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:59.874827 4114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "828f459a62484b30a50b2842c50bcec2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43535 } }
I20260807 19:42:59.877884 4115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "828f459a62484b30a50b2842c50bcec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "828f459a62484b30a50b2842c50bcec2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43535 } } }
I20260807 19:42:59.878263 4115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:59.879956 4119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:59.879796 4116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 828f459a62484b30a50b2842c50bcec2. Latest consensus state: current_term: 1 leader_uuid: "828f459a62484b30a50b2842c50bcec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "828f459a62484b30a50b2842c50bcec2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43535 } } }
I20260807 19:42:59.880180 4116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:59.886673 4119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:59.889591 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:59.869743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:57293 (local address 127.0.0.1:43535)
0807 19:42:59.869925 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:59.869931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:59.870324 (+ 393us) server_negotiation.cc:407] Connection header received
0807 19:42:59.870454 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:59.870461 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:59.870545 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:59.870646 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:59.872227 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.873702 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.875614 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.876340 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.876511 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:59.880786 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:59.880796 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:59.880809 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:59.880871 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:59.885247 (+ 4376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:59.886245 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:59.886250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:59.886253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:59.886432 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:59.887034 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:59.887039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:59.887043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:59.887481 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:59.887808 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:59.889078 (+ 1270us) server_negotiation.cc:299] Negotiation successful
0807 19:42:59.889253 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:42:59.897395 4119 catalog_manager.cc:1347] Generated new cluster ID: a2139b1e49b144d9bc5ba522f0806908
I20260807 19:42:59.897498 4119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:59.936414 4119 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:59.937759 4119 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:59.946985 4119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 828f459a62484b30a50b2842c50bcec2: Generated new TSK 0
I20260807 19:42:59.947872 4119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:59.959473 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4036
I20260807 19:42:59.960664 4098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:00.056838 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 4036
W20260807 19:43:00.077492 4021 connection.cc:535] client connection to 127.0.0.1:43535 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43535): Connection reset by peer (error 104)
2026-08-07T19:43:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2166 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:00 dist-test-slave-07c2 krb5kdc[4137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:00 dist-test-slave-07c2 krb5kdc[4137](info): set up 2 sockets
Aug 07 19:43:00 dist-test-slave-07c2 krb5kdc[4137](info): commencing operation
krb5kdc: starting...
W20260807 19:43:01.198076 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:43:01 dist-test-slave-07c2 krb5kdc[4137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:43:01.544970 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36211
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:01.884502 4150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:01.884786 4150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:01.884879 4150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:01.891865 4150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:01.891968 4150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:01.892015 4150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:01.892086 4150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:01.892123 4150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:01.901610 4150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36211
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:01.902875 4150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:01.904131 4150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:01.915723 4155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:01.915805 4156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:01.915756 4158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:01.916951 4150 server_base.cc:1034] running on GCE node
I20260807 19:43:01.917888 4150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:01.919555 4150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:01.920801 4150 hybrid_clock.cc:648] HybridClock initialized: now 1786131781920747 us; error 75 us; skew 500 ppm
I20260807 19:43:01.921147 4150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:01 dist-test-slave-07c2 krb5kdc[4137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:01.924388 4150 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:43:01.925778 4150 webserver.cc:460] Webserver started at http://127.0.0.1:37113/ using document root <none> and password file <none>
I20260807 19:43:01.926472 4150 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:01.926578 4150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:01.926838 4150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:01.929795 4150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "ef2f8b5eba2c404fab621b78c1b46a29"
format_stamp: "Formatted at 2026-08-07 19:43:01 on dist-test-slave-07c2"
server_key: "9086c84010bdd787e0e8dc6524ad2d15"
server_key_iv: "d8a411fded1f9892f662d6890fdbd0bd"
server_key_version: "encryptionkey@0"
I20260807 19:43:01.930491 4150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "ef2f8b5eba2c404fab621b78c1b46a29"
format_stamp: "Formatted at 2026-08-07 19:43:01 on dist-test-slave-07c2"
server_key: "9086c84010bdd787e0e8dc6524ad2d15"
server_key_iv: "d8a411fded1f9892f662d6890fdbd0bd"
server_key_version: "encryptionkey@0"
I20260807 19:43:01.935527 4150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:43:01.938807 4165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:01.940124 4150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:43:01.940310 4150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "ef2f8b5eba2c404fab621b78c1b46a29"
format_stamp: "Formatted at 2026-08-07 19:43:01 on dist-test-slave-07c2"
server_key: "9086c84010bdd787e0e8dc6524ad2d15"
server_key_iv: "d8a411fded1f9892f662d6890fdbd0bd"
server_key_version: "encryptionkey@0"
I20260807 19:43:01.940506 4150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:01.960737 4150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:01.965047 4150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:01.965298 4150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:01.984432 4150 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36211
I20260807 19:43:01.984464 4217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36211 every 8 connection(s)
I20260807 19:43:01.985852 4150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:01.987879 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 4150
I20260807 19:43:01.988057 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:43:01.988662 14401 external_mini_cluster.cc:1428] Setting key baace26a3a97fdadcac2f64f0e87073f
I20260807 19:43:01.990526 4218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:02.001971 4222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:01.990449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:33009 (local address 127.0.0.1:36211)
0807 19:43:01.991323 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:01.991340 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:01.991416 (+ 76us) server_negotiation.cc:407] Connection header received
0807 19:43:01.993012 (+ 1596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:01.993032 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:01.993415 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:01.993748 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:01.994345 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.995762 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.997594 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.998224 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.998347 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:01.999672 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:01.999703 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:01.999716 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:02.000270 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:02.000347 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:02.000370 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:02.000505 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:02.000830 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 19:43:02.001025 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":428,"threads_started":1}
I20260807 19:43:02.003201 4218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29: Bootstrap starting.
I20260807 19:43:02.005762 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:43:02.006500 4218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:02.007781 4218 log.cc:824] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:02.011039 4218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29: No bootstrap required, opened a new log
I20260807 19:43:02.018101 4222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:02.011029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:48245 (local address 127.0.0.1:36211)
0807 19:43:02.011237 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:02.011243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:02.011809 (+ 566us) server_negotiation.cc:407] Connection header received
0807 19:43:02.011938 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:02.011943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:02.012013 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:02.012129 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:02.012616 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.013825 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.015069 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.015813 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.015953 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:02.017136 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:02.017143 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:02.017150 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:02.017176 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:02.017234 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:02.017255 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:02.017374 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:02.017683 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 19:43:02.017740 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:43:02.020336 4218 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f8b5eba2c404fab621b78c1b46a29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36211 } }
I20260807 19:43:02.020686 4218 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:02.020777 4218 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2f8b5eba2c404fab621b78c1b46a29, State: Initialized, Role: FOLLOWER
I20260807 19:43:02.021346 4218 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f8b5eba2c404fab621b78c1b46a29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36211 } }
I20260807 19:43:02.021611 4218 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:02.021783 4218 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:02.021963 4218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:02.024194 4218 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f8b5eba2c404fab621b78c1b46a29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36211 } }
I20260807 19:43:02.024670 4218 leader_election.cc:302] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef2f8b5eba2c404fab621b78c1b46a29; no voters:
I20260807 19:43:02.025213 4218 leader_election.cc:288] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:02.025363 4229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:02.026088 4229 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [term 1 LEADER]: Becoming Leader. State: Replica: ef2f8b5eba2c404fab621b78c1b46a29, State: Running, Role: LEADER
I20260807 19:43:02.026636 4229 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f8b5eba2c404fab621b78c1b46a29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36211 } }
I20260807 19:43:02.027400 4218 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:02.029372 4230 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef2f8b5eba2c404fab621b78c1b46a29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f8b5eba2c404fab621b78c1b46a29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36211 } } }
I20260807 19:43:02.029740 4230 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:02.030277 4231 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef2f8b5eba2c404fab621b78c1b46a29. Latest consensus state: current_term: 1 leader_uuid: "ef2f8b5eba2c404fab621b78c1b46a29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef2f8b5eba2c404fab621b78c1b46a29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36211 } } }
I20260807 19:43:02.030604 4231 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:02.031531 4234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:02.036849 4234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:02.045938 4234 catalog_manager.cc:1347] Generated new cluster ID: c54c314e0d284a9db82ae2b7ed241797
I20260807 19:43:02.046082 4234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:02.060310 4234 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:02.062233 4234 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:02.071146 4234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef2f8b5eba2c404fab621b78c1b46a29: Generated new TSK 0
I20260807 19:43:02.072049 4234 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:43:02.077708 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4150
W20260807 19:43:02.078987 4213 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.85:33009
W20260807 19:43:02.079824 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4150: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:43:02.079988 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 4150
W20260807 19:43:02.100546 4135 connection.cc:535] client connection to 127.0.0.1:36211 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36211): Connection reset by peer (error 104)
2026-08-07T19:43:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2027 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:02 dist-test-slave-07c2 krb5kdc[4251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:02 dist-test-slave-07c2 krb5kdc[4251](info): set up 2 sockets
Aug 07 19:43:02 dist-test-slave-07c2 krb5kdc[4251](info): commencing operation
krb5kdc: starting...
W20260807 19:43:03.233120 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:43:03 dist-test-slave-07c2 krb5kdc[4251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:43:03.566423 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35915
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:03.916152 4266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:03.916451 4266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:03.916661 4266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:03.924031 4266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:03.924170 4266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:03.924279 4266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:03.924384 4266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:03.924463 4266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:03.934422 4266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35915
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:03.935758 4266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:03.937060 4266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:03.949375 4273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:03.949429 4275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:03.949455 4272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:03.950402 4266 server_base.cc:1034] running on GCE node
I20260807 19:43:03.951109 4266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:03.952515 4266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:03.953727 4266 hybrid_clock.cc:648] HybridClock initialized: now 1786131783953678 us; error 35 us; skew 500 ppm
I20260807 19:43:03.954044 4266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:03 dist-test-slave-07c2 krb5kdc[4251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131783, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:03.960822 4266 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:43:03.962451 4266 webserver.cc:460] Webserver started at http://127.0.0.1:35031/ using document root <none> and password file <none>
I20260807 19:43:03.963156 4266 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:03.963299 4266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:03.963665 4266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:03.966666 4266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "4e8e18d992b544ef874148e4f1285246"
format_stamp: "Formatted at 2026-08-07 19:43:03 on dist-test-slave-07c2"
server_key: "c68365748ad3aae5092edf5c9330a593"
server_key_iv: "8def228d507028f205c49e5b2858a774"
server_key_version: "encryptionkey@0"
I20260807 19:43:03.967499 4266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "4e8e18d992b544ef874148e4f1285246"
format_stamp: "Formatted at 2026-08-07 19:43:03 on dist-test-slave-07c2"
server_key: "c68365748ad3aae5092edf5c9330a593"
server_key_iv: "8def228d507028f205c49e5b2858a774"
server_key_version: "encryptionkey@0"
I20260807 19:43:03.972573 4266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:43:03.975955 4282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:03.977250 4266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:43:03.977428 4266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "4e8e18d992b544ef874148e4f1285246"
format_stamp: "Formatted at 2026-08-07 19:43:03 on dist-test-slave-07c2"
server_key: "c68365748ad3aae5092edf5c9330a593"
server_key_iv: "8def228d507028f205c49e5b2858a774"
server_key_version: "encryptionkey@0"
I20260807 19:43:03.977629 4266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:03.983446 4266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:03.987277 4266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:03.987612 4266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:04.006249 4266 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35915
I20260807 19:43:04.006305 4335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35915 every 8 connection(s)
I20260807 19:43:04.007740 4266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:04.010816 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 4266
I20260807 19:43:04.011018 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:43:04.011473 4336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:04.011585 14401 external_mini_cluster.cc:1428] Setting key eca94f5ea0f980cf2304f576b91a8fb9
I20260807 19:43:04.020850 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:04.013384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:52765 (local address 127.0.0.1:35915)
0807 19:43:04.014231 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:04.014246 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:04.014298 (+ 52us) server_negotiation.cc:407] Connection header received
0807 19:43:04.016614 (+ 2316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:04.016641 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:04.017083 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:04.017504 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:04.018571 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:04.018586 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:04.018601 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:04.018625 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:04.018699 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:04.018730 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:04.018962 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:04.019096 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 19:43:04.019330 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":324,"threads_started":1}
I20260807 19:43:04.024487 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:43:04.026414 4336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246: Bootstrap starting.
I20260807 19:43:04.029773 4336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:04.030949 4336 log.cc:824] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:04.031438 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:04.028865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:45321 (local address 127.0.0.1:35915)
0807 19:43:04.029011 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:04.029015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:04.029476 (+ 461us) server_negotiation.cc:407] Connection header received
0807 19:43:04.029664 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:04.029669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:04.029729 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:04.029838 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:04.030752 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:04.030757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:04.030763 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:04.030785 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:04.030825 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:04.030840 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:04.030960 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:04.031068 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 19:43:04.031126 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:43:04.033811 4336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246: No bootstrap required, opened a new log
I20260807 19:43:04.041420 4336 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e8e18d992b544ef874148e4f1285246" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35915 } }
I20260807 19:43:04.041807 4336 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:04.041927 4336 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e8e18d992b544ef874148e4f1285246, State: Initialized, Role: FOLLOWER
I20260807 19:43:04.042500 4336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e8e18d992b544ef874148e4f1285246" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35915 } }
I20260807 19:43:04.042757 4336 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:04.042917 4336 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:04.043083 4336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:04.045533 4336 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e8e18d992b544ef874148e4f1285246" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35915 } }
I20260807 19:43:04.046079 4336 leader_election.cc:302] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e8e18d992b544ef874148e4f1285246; no voters:
I20260807 19:43:04.046747 4336 leader_election.cc:288] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:04.046903 4346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:04.047679 4346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [term 1 LEADER]: Becoming Leader. State: Replica: 4e8e18d992b544ef874148e4f1285246, State: Running, Role: LEADER
I20260807 19:43:04.048193 4346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e8e18d992b544ef874148e4f1285246" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35915 } }
I20260807 19:43:04.048969 4336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:04.051908 4349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e8e18d992b544ef874148e4f1285246. Latest consensus state: current_term: 1 leader_uuid: "4e8e18d992b544ef874148e4f1285246" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e8e18d992b544ef874148e4f1285246" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35915 } } }
I20260807 19:43:04.052407 4349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:04.051944 4347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e8e18d992b544ef874148e4f1285246" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e8e18d992b544ef874148e4f1285246" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35915 } } }
I20260807 19:43:04.053066 4347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:04.059953 4355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:04.064347 4355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:04.072213 4355 catalog_manager.cc:1347] Generated new cluster ID: ef349442419e4cf9a1a7e77c5faed284
I20260807 19:43:04.072329 4355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:04.081507 4355 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:04.082089 4355 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:04.088936 4355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e8e18d992b544ef874148e4f1285246: Generated new TSK 0
I20260807 19:43:04.089957 4355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:43:04.104146 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4266
W20260807 19:43:04.105700 4331 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.85:52765
W20260807 19:43:04.106429 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4266: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:43:04.106600 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 4266
2026-08-07T19:43:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2020 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:04 dist-test-slave-07c2 krb5kdc[4369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:04 dist-test-slave-07c2 krb5kdc[4369](info): set up 2 sockets
Aug 07 19:43:04 dist-test-slave-07c2 krb5kdc[4369](info): commencing operation
krb5kdc: starting...
W20260807 19:43:05.318861 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:43:05 dist-test-slave-07c2 krb5kdc[4369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:43:05.644495 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:39257
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:39257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:05.984110 4382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:05.984418 4382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:05.984591 4382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:05.991891 4382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:05.992043 4382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:05.992146 4382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:05.992257 4382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:05.992322 4382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:06.002198 4382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:39257
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:39257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:06.003703 4382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:06.005043 4382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:06.017189 4388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:06.017212 4390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:06.017277 4387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:06.017819 4382 server_base.cc:1034] running on GCE node
I20260807 19:43:06.018695 4382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:06.020126 4382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:06.021363 4382 hybrid_clock.cc:648] HybridClock initialized: now 1786131786021334 us; error 51 us; skew 500 ppm
I20260807 19:43:06.021725 4382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:06 dist-test-slave-07c2 krb5kdc[4369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131786, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:06.024958 4382 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:43:06.026515 4382 webserver.cc:460] Webserver started at http://127.14.16.126:37575/ using document root <none> and password file <none>
I20260807 19:43:06.027184 4382 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:06.027349 4382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:06.027657 4382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:06.030587 4382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "2ee63ea2f3b34b429ab66c22bc4c33e3"
format_stamp: "Formatted at 2026-08-07 19:43:06 on dist-test-slave-07c2"
server_key: "baccd8224ee527e0704f7dc578008719"
server_key_iv: "e6ea1cdfcacc0b42fb8d841db15c0344"
server_key_version: "encryptionkey@0"
I20260807 19:43:06.031412 4382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "2ee63ea2f3b34b429ab66c22bc4c33e3"
format_stamp: "Formatted at 2026-08-07 19:43:06 on dist-test-slave-07c2"
server_key: "baccd8224ee527e0704f7dc578008719"
server_key_iv: "e6ea1cdfcacc0b42fb8d841db15c0344"
server_key_version: "encryptionkey@0"
I20260807 19:43:06.036582 4382 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:43:06.039940 4399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:06.041180 4382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:43:06.041373 4382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "2ee63ea2f3b34b429ab66c22bc4c33e3"
format_stamp: "Formatted at 2026-08-07 19:43:06 on dist-test-slave-07c2"
server_key: "baccd8224ee527e0704f7dc578008719"
server_key_iv: "e6ea1cdfcacc0b42fb8d841db15c0344"
server_key_version: "encryptionkey@0"
I20260807 19:43:06.041566 4382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:06.060230 4382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:06.064724 4382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:06.065033 4382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:06.084146 4382 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:39257
I20260807 19:43:06.084172 4453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:39257 every 8 connection(s)
I20260807 19:43:06.085510 4382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:06.087708 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 4382
I20260807 19:43:06.087937 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:43:06.088500 14401 external_mini_cluster.cc:1428] Setting key 90e6f20864cf0dca5a6557ef522aad33
I20260807 19:43:06.089954 4454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:43:06 dist-test-slave-07c2 krb5kdc[4369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:43:06.104313 4454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3: Bootstrap starting.
I20260807 19:43:06.107522 4454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:06.108582 4454 log.cc:824] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:06.111656 4454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3: No bootstrap required, opened a new log
I20260807 19:43:06.111819 4457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:06.090423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51715 (local address 127.14.16.126:39257)
0807 19:43:06.091208 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:06.091219 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:06.091259 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:43:06.092847 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:06.092866 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:06.093236 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:06.093551 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:06.095423 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.096736 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.098468 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.099120 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.099237 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:06.104326 (+ 5089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:06.104334 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:06.104348 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:06.104407 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:06.107547 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:06.108378 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:06.108384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:06.108391 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:06.108585 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:06.109069 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:06.109078 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:06.109080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:06.109581 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:06.109959 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:06.110439 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 19:43:06.110782 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":360,"threads_started":1}
I20260807 19:43:06.115562 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:43:06.119820 4454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee63ea2f3b34b429ab66c22bc4c33e3" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39257 } }
I20260807 19:43:06.120329 4454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:06.120450 4454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ee63ea2f3b34b429ab66c22bc4c33e3, State: Initialized, Role: FOLLOWER
I20260807 19:43:06.121079 4454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee63ea2f3b34b429ab66c22bc4c33e3" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39257 } }
I20260807 19:43:06.121373 4454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:06.121524 4454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:06.121719 4454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:06.124577 4454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee63ea2f3b34b429ab66c22bc4c33e3" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39257 } }
I20260807 19:43:06.125105 4454 leader_election.cc:302] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ee63ea2f3b34b429ab66c22bc4c33e3; no voters:
I20260807 19:43:06.125771 4454 leader_election.cc:288] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:06.125913 4465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:06.126632 4465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [term 1 LEADER]: Becoming Leader. State: Replica: 2ee63ea2f3b34b429ab66c22bc4c33e3, State: Running, Role: LEADER
I20260807 19:43:06.127345 4465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee63ea2f3b34b429ab66c22bc4c33e3" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39257 } }
I20260807 19:43:06.129050 4454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:06.130447 4466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ee63ea2f3b34b429ab66c22bc4c33e3. Latest consensus state: current_term: 1 leader_uuid: "2ee63ea2f3b34b429ab66c22bc4c33e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee63ea2f3b34b429ab66c22bc4c33e3" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39257 } } }
I20260807 19:43:06.130982 4466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:06.130844 4467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ee63ea2f3b34b429ab66c22bc4c33e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee63ea2f3b34b429ab66c22bc4c33e3" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39257 } } }
I20260807 19:43:06.131182 4467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:06.132730 4470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:06.137668 4470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:06.138518 4457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:06.119953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48481 (local address 127.14.16.126:39257)
0807 19:43:06.120147 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:06.120152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:06.120605 (+ 453us) server_negotiation.cc:407] Connection header received
0807 19:43:06.120887 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:06.120892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:06.120971 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:06.121089 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:06.122623 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.123820 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.125066 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.125864 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.125988 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:06.129344 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:06.129351 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:06.129355 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:06.129394 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:06.133583 (+ 4189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:06.134760 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:06.134766 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:06.134772 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:06.135015 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:06.135803 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:06.135811 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:06.135817 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:06.136350 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:06.136709 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:06.137669 (+ 960us) server_negotiation.cc:299] Negotiation successful
0807 19:43:06.137924 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:43:06.148217 4470 catalog_manager.cc:1347] Generated new cluster ID: 343dc4b6c91b42418942fa856ecf7564
I20260807 19:43:06.148345 4470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:06.155625 4470 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:06.156749 4470 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:06.165270 4470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ee63ea2f3b34b429ab66c22bc4c33e3: Generated new TSK 0
I20260807 19:43:06.166268 4470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:43:06.178885 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4382
I20260807 19:43:06.180188 4449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:06.270888 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 4382
W20260807 19:43:06.290874 4367 connection.cc:535] client connection to 127.14.16.126:39257 recv error: Network error: failed to read from TLS socket (remote: 127.14.16.126:39257): Connection reset by peer (error 104)
2026-08-07T19:43:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2164 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:06 dist-test-slave-07c2 krb5kdc[4488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:06 dist-test-slave-07c2 krb5kdc[4488](info): set up 2 sockets
Aug 07 19:43:06 dist-test-slave-07c2 krb5kdc[4488](info): commencing operation
krb5kdc: starting...
W20260807 19:43:07.410281 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.008s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:43:07 dist-test-slave-07c2 krb5kdc[4488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:43:07.752270 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:36085
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:36085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:08.087375 4502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:08.087729 4502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:08.087903 4502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:08.095839 4502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:08.095993 4502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:08.096081 4502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:08.096182 4502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:08.096251 4502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:08.106027 4502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:36085
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:36085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:08.107400 4502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:08.108750 4502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:08.120477 4507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:08.120456 4508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:08.120743 4510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:08.121661 4502 server_base.cc:1034] running on GCE node
I20260807 19:43:08.122367 4502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:08.123752 4502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:08.124985 4502 hybrid_clock.cc:648] HybridClock initialized: now 1786131788124945 us; error 32 us; skew 500 ppm
I20260807 19:43:08.125339 4502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:08 dist-test-slave-07c2 krb5kdc[4488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131788, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:08.128922 4502 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:43:08.130414 4502 webserver.cc:460] Webserver started at http://127.14.16.126:46883/ using document root <none> and password file <none>
I20260807 19:43:08.131074 4502 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:08.131229 4502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:08.131558 4502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:08.134464 4502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "f98498b225ad45ab9d95e653b7bf2cf8"
format_stamp: "Formatted at 2026-08-07 19:43:08 on dist-test-slave-07c2"
server_key: "d62e3e424669ad9fbe7ca099003c6dcb"
server_key_iv: "84e544fe2d457592d818c95885337315"
server_key_version: "encryptionkey@0"
I20260807 19:43:08.135224 4502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "f98498b225ad45ab9d95e653b7bf2cf8"
format_stamp: "Formatted at 2026-08-07 19:43:08 on dist-test-slave-07c2"
server_key: "d62e3e424669ad9fbe7ca099003c6dcb"
server_key_iv: "84e544fe2d457592d818c95885337315"
server_key_version: "encryptionkey@0"
I20260807 19:43:08.140480 4502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 19:43:08.144151 4517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:08.145411 4502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:43:08.145610 4502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "f98498b225ad45ab9d95e653b7bf2cf8"
format_stamp: "Formatted at 2026-08-07 19:43:08 on dist-test-slave-07c2"
server_key: "d62e3e424669ad9fbe7ca099003c6dcb"
server_key_iv: "84e544fe2d457592d818c95885337315"
server_key_version: "encryptionkey@0"
I20260807 19:43:08.145823 4502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:08.160322 4502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:08.164762 4502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:08.165097 4502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:08.185385 4502 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:36085
I20260807 19:43:08.185392 4572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:36085 every 8 connection(s)
I20260807 19:43:08.186870 4502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:08.190490 4573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:08.193123 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 4502
I20260807 19:43:08.193318 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:43:08.193843 14401 external_mini_cluster.cc:1428] Setting key fc0414686c4387b594568ab32a1647e1
I20260807 19:43:08.202623 4573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8: Bootstrap starting.
I20260807 19:43:08.206298 4573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:08.207398 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:08.195662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48621 (local address 127.14.16.126:36085)
0807 19:43:08.196492 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:08.196504 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:08.196547 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:43:08.198210 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:08.198233 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:08.198605 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:08.198925 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:08.199584 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.200952 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.202800 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.203527 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.203667 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:08.204998 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:08.205005 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:08.205017 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:08.205534 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:08.205607 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:08.205630 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:08.205767 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:08.206079 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 19:43:08.206252 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":387,"threads_started":1}
I20260807 19:43:08.207530 4573 log.cc:824] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:08.210690 4573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8: No bootstrap required, opened a new log
I20260807 19:43:08.211685 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:43:08.221732 4573 raft_consensus.cc:348] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f98498b225ad45ab9d95e653b7bf2cf8" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36085 } }
I20260807 19:43:08.222239 4573 raft_consensus.cc:374] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:08.222404 4573 raft_consensus.cc:729] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f98498b225ad45ab9d95e653b7bf2cf8, State: Initialized, Role: FOLLOWER
I20260807 19:43:08.223153 4573 consensus_queue.cc:260] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f98498b225ad45ab9d95e653b7bf2cf8" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36085 } }
I20260807 19:43:08.223536 4573 raft_consensus.cc:388] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:08.223699 4573 raft_consensus.cc:482] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:08.223945 4573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:08.224833 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:08.216757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36959 (local address 127.14.16.126:36085)
0807 19:43:08.216947 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:08.216952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:08.217474 (+ 522us) server_negotiation.cc:407] Connection header received
0807 19:43:08.217609 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:08.217615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:08.217691 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:08.217807 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:08.218291 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.219294 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.221521 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.222202 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.222345 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:08.223885 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:08.223897 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:08.223902 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:08.223927 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:08.223999 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:08.224024 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:08.224154 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:08.224419 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 19:43:08.224477 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:43:08.227602 4573 raft_consensus.cc:504] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f98498b225ad45ab9d95e653b7bf2cf8" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36085 } }
I20260807 19:43:08.228130 4573 leader_election.cc:302] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f98498b225ad45ab9d95e653b7bf2cf8; no voters:
I20260807 19:43:08.228688 4573 leader_election.cc:288] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:08.228826 4584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:08.229516 4584 raft_consensus.cc:686] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [term 1 LEADER]: Becoming Leader. State: Replica: f98498b225ad45ab9d95e653b7bf2cf8, State: Running, Role: LEADER
I20260807 19:43:08.230018 4584 consensus_queue.cc:237] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f98498b225ad45ab9d95e653b7bf2cf8" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36085 } }
I20260807 19:43:08.230813 4573 sys_catalog.cc:564] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:08.232957 4586 sys_catalog.cc:455] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f98498b225ad45ab9d95e653b7bf2cf8. Latest consensus state: current_term: 1 leader_uuid: "f98498b225ad45ab9d95e653b7bf2cf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f98498b225ad45ab9d95e653b7bf2cf8" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36085 } } }
I20260807 19:43:08.233405 4586 sys_catalog.cc:458] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:08.232957 4585 sys_catalog.cc:455] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f98498b225ad45ab9d95e653b7bf2cf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f98498b225ad45ab9d95e653b7bf2cf8" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 36085 } } }
I20260807 19:43:08.233809 4585 sys_catalog.cc:458] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:08.234966 4591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:08.240417 4591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:08.249289 4591 catalog_manager.cc:1347] Generated new cluster ID: 1b95eb1c95314b1d900921ccfa561950
I20260807 19:43:08.249413 4591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:08.279438 4591 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:08.280803 4591 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:08.287985 4591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f98498b225ad45ab9d95e653b7bf2cf8: Generated new TSK 0
I20260807 19:43:08.288828 4591 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:43:08.299608 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4502
W20260807 19:43:08.300807 4568 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48621
W20260807 19:43:08.301648 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4502: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:43:08.301827 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 4502
W20260807 19:43:08.321455 4485 connection.cc:535] client connection to 127.14.16.126:36085 recv error: Network error: failed to read from TLS socket (remote: 127.14.16.126:36085): Connection reset by peer (error 104)
2026-08-07T19:43:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2030 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:08 dist-test-slave-07c2 krb5kdc[4605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:08 dist-test-slave-07c2 krb5kdc[4605](info): set up 2 sockets
Aug 07 19:43:08 dist-test-slave-07c2 krb5kdc[4605](info): commencing operation
krb5kdc: starting...
W20260807 19:43:09.450519 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:43:09 dist-test-slave-07c2 krb5kdc[4605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:43:09.774725 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:39091
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:46371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:39091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:10.127370 4618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:10.127743 4618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:10.127915 4618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:10.135535 4618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:10.135687 4618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:10.135802 4618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:10.135921 4618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:10.136005 4618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:10.146155 4618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:46371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:39091
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:39091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:10.147505 4618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:10.148871 4618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:10.160776 4624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:10.160733 4627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:10.160669 4625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:10.161379 4618 server_base.cc:1034] running on GCE node
I20260807 19:43:10.162148 4618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:10.163650 4618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:10.164909 4618 hybrid_clock.cc:648] HybridClock initialized: now 1786131790164849 us; error 55 us; skew 500 ppm
I20260807 19:43:10.165293 4618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:10 dist-test-slave-07c2 krb5kdc[4605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131790, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:10.175019 4618 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:43:10.176781 4618 webserver.cc:460] Webserver started at http://127.14.16.126:32939/ using document root <none> and password file <none>
I20260807 19:43:10.177465 4618 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:10.177579 4618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:10.177836 4618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:10.180878 4618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "fab4a1e2d3b04fb6aa88d1f683fa83d1"
format_stamp: "Formatted at 2026-08-07 19:43:10 on dist-test-slave-07c2"
server_key: "e32e6957a8315399a60e8d82df3369ed"
server_key_iv: "529149faae380dfd75ef240b243f345e"
server_key_version: "encryptionkey@0"
I20260807 19:43:10.181591 4618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "fab4a1e2d3b04fb6aa88d1f683fa83d1"
format_stamp: "Formatted at 2026-08-07 19:43:10 on dist-test-slave-07c2"
server_key: "e32e6957a8315399a60e8d82df3369ed"
server_key_iv: "529149faae380dfd75ef240b243f345e"
server_key_version: "encryptionkey@0"
I20260807 19:43:10.186825 4618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 19:43:10.190384 4634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:10.191895 4618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:43:10.192090 4618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "fab4a1e2d3b04fb6aa88d1f683fa83d1"
format_stamp: "Formatted at 2026-08-07 19:43:10 on dist-test-slave-07c2"
server_key: "e32e6957a8315399a60e8d82df3369ed"
server_key_iv: "529149faae380dfd75ef240b243f345e"
server_key_version: "encryptionkey@0"
I20260807 19:43:10.192315 4618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:10.198021 4618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:10.202250 4618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:10.202555 4618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:10.221599 4618 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:39091
I20260807 19:43:10.221638 4687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:39091 every 8 connection(s)
I20260807 19:43:10.223012 4618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:10.225131 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 4618
I20260807 19:43:10.225317 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:43:10.225785 14401 external_mini_cluster.cc:1428] Setting key c904437d821b79b38c24a7a8f51943c7
I20260807 19:43:10.227077 4688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:10.233160 4691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:10.227556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43099 (local address 127.14.16.126:39091)
0807 19:43:10.228338 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:10.228349 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:10.228390 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:43:10.229909 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:10.229934 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:10.230246 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:10.230555 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:10.231636 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:10.231642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:10.231654 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:10.231668 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:10.231764 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:10.231784 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:10.231946 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:10.232071 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 19:43:10.232277 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":359,"threads_started":1}
I20260807 19:43:10.236650 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:43:10.240339 4688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1: Bootstrap starting.
I20260807 19:43:10.243944 4688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:10.244325 4691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:10.241776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60217 (local address 127.14.16.126:39091)
0807 19:43:10.241970 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:10.241977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:10.242507 (+ 530us) server_negotiation.cc:407] Connection header received
0807 19:43:10.242631 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:10.242636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:10.242692 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:10.242794 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:10.243683 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:10.243691 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:10.243698 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:10.243718 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:10.243767 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:10.243789 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:10.243918 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:10.243975 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 19:43:10.244033 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:43:10.245162 4688 log.cc:824] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:10.247807 4688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1: No bootstrap required, opened a new log
I20260807 19:43:10.254776 4688 raft_consensus.cc:348] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab4a1e2d3b04fb6aa88d1f683fa83d1" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39091 } }
I20260807 19:43:10.255134 4688 raft_consensus.cc:374] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:10.255254 4688 raft_consensus.cc:729] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab4a1e2d3b04fb6aa88d1f683fa83d1, State: Initialized, Role: FOLLOWER
I20260807 19:43:10.255862 4688 consensus_queue.cc:260] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab4a1e2d3b04fb6aa88d1f683fa83d1" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39091 } }
I20260807 19:43:10.256084 4688 raft_consensus.cc:388] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:10.256247 4688 raft_consensus.cc:482] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:10.256430 4688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:10.258611 4688 raft_consensus.cc:504] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab4a1e2d3b04fb6aa88d1f683fa83d1" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39091 } }
I20260807 19:43:10.259114 4688 leader_election.cc:302] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fab4a1e2d3b04fb6aa88d1f683fa83d1; no voters:
I20260807 19:43:10.259742 4688 leader_election.cc:288] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:10.259874 4698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:10.260609 4698 raft_consensus.cc:686] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [term 1 LEADER]: Becoming Leader. State: Replica: fab4a1e2d3b04fb6aa88d1f683fa83d1, State: Running, Role: LEADER
I20260807 19:43:10.261113 4698 consensus_queue.cc:237] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab4a1e2d3b04fb6aa88d1f683fa83d1" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39091 } }
I20260807 19:43:10.261857 4688 sys_catalog.cc:564] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:10.264180 4700 sys_catalog.cc:455] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fab4a1e2d3b04fb6aa88d1f683fa83d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab4a1e2d3b04fb6aa88d1f683fa83d1" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39091 } } }
I20260807 19:43:10.264709 4700 sys_catalog.cc:458] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:10.265305 4701 sys_catalog.cc:455] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fab4a1e2d3b04fb6aa88d1f683fa83d1. Latest consensus state: current_term: 1 leader_uuid: "fab4a1e2d3b04fb6aa88d1f683fa83d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab4a1e2d3b04fb6aa88d1f683fa83d1" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 39091 } } }
I20260807 19:43:10.266294 4701 sys_catalog.cc:458] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:10.271734 4707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:10.275874 4707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:10.284556 4707 catalog_manager.cc:1347] Generated new cluster ID: b14a6b22a47046e7b445fbc63f7c83cf
I20260807 19:43:10.284667 4707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:10.292972 4707 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:10.293447 4707 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:10.300856 4707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fab4a1e2d3b04fb6aa88d1f683fa83d1: Generated new TSK 0
I20260807 19:43:10.301836 4707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:43:10.309590 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4618
W20260807 19:43:10.310889 4683 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43099
W20260807 19:43:10.311601 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4618: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:43:10.311766 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 4618
2026-08-07T19:43:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2009 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:10 dist-test-slave-07c2 krb5kdc[4720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:10 dist-test-slave-07c2 krb5kdc[4720](info): set up 2 sockets
Aug 07 19:43:10 dist-test-slave-07c2 krb5kdc[4720](info): commencing operation
krb5kdc: starting...
W20260807 19:43:11.458223 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:43:11 dist-test-slave-07c2 krb5kdc[4720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:43:11.794582 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:33843
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:34383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:33843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:12.134711 4734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:12.135064 4734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:12.135234 4734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:12.142411 4734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:12.142570 4734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:12.142674 4734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:12.142787 4734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:12.142864 4734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:12.152730 4734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:34383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:33843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:33843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:12.154018 4734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:12.155421 4734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:12.167495 4742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:12.167636 4739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:12.167622 4740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:12.169044 4734 server_base.cc:1034] running on GCE node
I20260807 19:43:12.169807 4734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:12.171195 4734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:12.172427 4734 hybrid_clock.cc:648] HybridClock initialized: now 1786131792172384 us; error 41 us; skew 500 ppm
I20260807 19:43:12.172791 4734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:12 dist-test-slave-07c2 krb5kdc[4720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131792, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:12.176214 4734 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:43:12.177691 4734 webserver.cc:460] Webserver started at http://127.14.16.126:41037/ using document root <none> and password file <none>
I20260807 19:43:12.178361 4734 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:12.178472 4734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:12.178745 4734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:12.181999 4734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "5f84385d222d461a8614129a060c43c0"
format_stamp: "Formatted at 2026-08-07 19:43:12 on dist-test-slave-07c2"
server_key: "58c0b8ed84afb6d6dfcfa7b93410b03c"
server_key_iv: "137907c1c8ab53bc3317e83abb88edae"
server_key_version: "encryptionkey@0"
I20260807 19:43:12.182760 4734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "5f84385d222d461a8614129a060c43c0"
format_stamp: "Formatted at 2026-08-07 19:43:12 on dist-test-slave-07c2"
server_key: "58c0b8ed84afb6d6dfcfa7b93410b03c"
server_key_iv: "137907c1c8ab53bc3317e83abb88edae"
server_key_version: "encryptionkey@0"
I20260807 19:43:12.188421 4734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260807 19:43:12.192018 4749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:12.193349 4734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:43:12.193528 4734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "5f84385d222d461a8614129a060c43c0"
format_stamp: "Formatted at 2026-08-07 19:43:12 on dist-test-slave-07c2"
server_key: "58c0b8ed84afb6d6dfcfa7b93410b03c"
server_key_iv: "137907c1c8ab53bc3317e83abb88edae"
server_key_version: "encryptionkey@0"
I20260807 19:43:12.193734 4734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:12.211872 4734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:12.216374 4734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:12.216678 4734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:12.235517 4734 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:33843
I20260807 19:43:12.235549 4800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:33843 every 8 connection(s)
I20260807 19:43:12.236937 4734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:12.237488 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 4734
I20260807 19:43:12.237699 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:43:12.238327 14401 external_mini_cluster.cc:1428] Setting key 72ea92c7ae859cfcf5e58d931e3a9a16
I20260807 19:43:12.241324 4801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:43:12 dist-test-slave-07c2 krb5kdc[4720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:43:12.255303 4801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0: Bootstrap starting.
I20260807 19:43:12.259541 4801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:12.260619 4801 log.cc:824] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:12.262625 4804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:12.240304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48067 (local address 127.14.16.126:33843)
0807 19:43:12.241130 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:12.241141 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:12.241180 (+ 39us) server_negotiation.cc:407] Connection header received
0807 19:43:12.242820 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:12.242842 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:12.243192 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:12.243577 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:12.245330 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:12.246626 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:12.248502 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:12.249155 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:12.249282 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:12.254308 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:12.254319 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:12.254338 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:12.254385 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:12.258386 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:12.259075 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:12.259083 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:12.259091 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:12.259355 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:12.259786 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:12.259789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:12.259791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:12.260338 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:12.260655 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:12.261161 (+ 506us) server_negotiation.cc:299] Negotiation successful
0807 19:43:12.261654 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":389,"threads_started":1}
I20260807 19:43:12.263669 4801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0: No bootstrap required, opened a new log
I20260807 19:43:12.266230 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:43:12.271378 4801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f84385d222d461a8614129a060c43c0" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33843 } }
I20260807 19:43:12.271852 4801 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:12.271953 4801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f84385d222d461a8614129a060c43c0, State: Initialized, Role: FOLLOWER
I20260807 19:43:12.272541 4801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f84385d222d461a8614129a060c43c0" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33843 } }
I20260807 19:43:12.272778 4801 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:12.272933 4801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:12.273113 4801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:12.275666 4801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f84385d222d461a8614129a060c43c0" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33843 } }
I20260807 19:43:12.276197 4801 leader_election.cc:302] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f84385d222d461a8614129a060c43c0; no voters:
I20260807 19:43:12.276872 4801 leader_election.cc:288] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:12.276973 4811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:12.279274 4811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [term 1 LEADER]: Becoming Leader. State: Replica: 5f84385d222d461a8614129a060c43c0, State: Running, Role: LEADER
I20260807 19:43:12.279552 4801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:12.279968 4811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f84385d222d461a8614129a060c43c0" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33843 } }
I20260807 19:43:12.286096 4804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:12.270696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52099 (local address 127.14.16.126:33843)
0807 19:43:12.270878 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:12.270884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:12.271322 (+ 438us) server_negotiation.cc:407] Connection header received
0807 19:43:12.271507 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:12.271513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:12.271599 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:12.271721 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:12.273561 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:12.274396 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:12.275711 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:12.276240 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:12.276377 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:12.279098 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:12.279111 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:12.279118 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:12.279166 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:12.282605 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:12.283372 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:12.283378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:12.283381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:12.283530 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:12.283912 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:12.283917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:12.283920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:12.284244 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:12.285420 (+ 1176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:12.285625 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 19:43:12.285800 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:43:12.289060 4814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f84385d222d461a8614129a060c43c0. Latest consensus state: current_term: 1 leader_uuid: "5f84385d222d461a8614129a060c43c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f84385d222d461a8614129a060c43c0" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33843 } } }
I20260807 19:43:12.289455 4814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:12.290872 4813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f84385d222d461a8614129a060c43c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f84385d222d461a8614129a060c43c0" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 33843 } } }
I20260807 19:43:12.291280 4813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:12.292371 4822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:12.298486 4822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:12.306703 4822 catalog_manager.cc:1347] Generated new cluster ID: bd211ecaf3d7481bb4f699aedaacc004
I20260807 19:43:12.306814 4822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:12.338027 4822 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:12.339452 4822 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:12.351843 4822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f84385d222d461a8614129a060c43c0: Generated new TSK 0
I20260807 19:43:12.352710 4822 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:43:12.361577 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4734
I20260807 19:43:12.362762 4796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:12.456211 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 4734
2026-08-07T19:43:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2145 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:12 dist-test-slave-07c2 krb5kdc[4834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:12 dist-test-slave-07c2 krb5kdc[4834](info): set up 2 sockets
Aug 07 19:43:12 dist-test-slave-07c2 krb5kdc[4834](info): commencing operation
krb5kdc: starting...
W20260807 19:43:13.618831 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:43:13 dist-test-slave-07c2 krb5kdc[4834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:43:13.944751 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:38969
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:43929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:38969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:14.285239 4848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:14.285549 4848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:14.285727 4848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:14.293286 4848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:14.293411 4848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:14.293525 4848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:14.293639 4848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:14.293704 4848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:14.303983 4848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:43929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:38969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:38969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:14.305344 4848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:14.306653 4848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:14.318730 4853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:14.318756 4854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:14.318657 4856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:14.319396 4848 server_base.cc:1034] running on GCE node
I20260807 19:43:14.320214 4848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:14.321591 4848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:14.322799 4848 hybrid_clock.cc:648] HybridClock initialized: now 1786131794322747 us; error 53 us; skew 500 ppm
I20260807 19:43:14.323113 4848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:14 dist-test-slave-07c2 krb5kdc[4834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131794, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:14.326386 4848 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:43:14.327907 4848 webserver.cc:460] Webserver started at http://127.14.16.126:44355/ using document root <none> and password file <none>
I20260807 19:43:14.328615 4848 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:14.328761 4848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:14.329070 4848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:14.332108 4848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "28d2baf1a8014a3e86e323ceedf38425"
format_stamp: "Formatted at 2026-08-07 19:43:14 on dist-test-slave-07c2"
server_key: "999b482367a2c6de60f2f3dab078d0df"
server_key_iv: "dfc9aca3d6203001e6096f819abb7b9f"
server_key_version: "encryptionkey@0"
I20260807 19:43:14.332877 4848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "28d2baf1a8014a3e86e323ceedf38425"
format_stamp: "Formatted at 2026-08-07 19:43:14 on dist-test-slave-07c2"
server_key: "999b482367a2c6de60f2f3dab078d0df"
server_key_iv: "dfc9aca3d6203001e6096f819abb7b9f"
server_key_version: "encryptionkey@0"
I20260807 19:43:14.338894 4848 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 19:43:14.342327 4863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:14.343736 4848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:43:14.343919 4848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "28d2baf1a8014a3e86e323ceedf38425"
format_stamp: "Formatted at 2026-08-07 19:43:14 on dist-test-slave-07c2"
server_key: "999b482367a2c6de60f2f3dab078d0df"
server_key_iv: "dfc9aca3d6203001e6096f819abb7b9f"
server_key_version: "encryptionkey@0"
I20260807 19:43:14.344125 4848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:14.357671 4848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:14.361513 4848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:14.361809 4848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:14.380823 4848 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:38969
I20260807 19:43:14.380856 4916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:38969 every 8 connection(s)
I20260807 19:43:14.382210 4848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:14.385763 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 4848
I20260807 19:43:14.385941 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:43:14.386096 4917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:14.386521 14401 external_mini_cluster.cc:1428] Setting key b3b162094d88ecf44ad8d9f09a52faf5
I20260807 19:43:14.399709 4920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:14.388344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46985 (local address 127.14.16.126:38969)
0807 19:43:14.389160 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:14.389174 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:14.389222 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:43:14.390962 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:14.390983 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:14.391415 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:14.391733 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:14.392317 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.393570 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.395414 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.396032 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.396168 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:14.397465 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:14.397472 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:14.397483 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:14.398012 (+ 529us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:14.398083 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:14.398109 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:14.398201 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:14.398448 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 19:43:14.398707 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":371,"threads_started":1}
I20260807 19:43:14.400190 4917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425: Bootstrap starting.
I20260807 19:43:14.403726 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:43:14.403715 4917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:14.404914 4917 log.cc:824] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:14.408329 4917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425: No bootstrap required, opened a new log
I20260807 19:43:14.415454 4920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:14.408912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56561 (local address 127.14.16.126:38969)
0807 19:43:14.409112 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:14.409127 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:14.409550 (+ 423us) server_negotiation.cc:407] Connection header received
0807 19:43:14.409677 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:14.409683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:14.409756 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:14.409863 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:14.410306 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.411462 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.412782 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.413530 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.413650 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:14.414695 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:14.414702 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:14.414706 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:14.414726 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:14.414779 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:14.414798 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:14.414894 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:14.415025 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 19:43:14.415130 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:43:14.417888 4917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d2baf1a8014a3e86e323ceedf38425" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38969 } }
I20260807 19:43:14.418216 4917 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:14.418288 4917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d2baf1a8014a3e86e323ceedf38425, State: Initialized, Role: FOLLOWER
I20260807 19:43:14.418761 4917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d2baf1a8014a3e86e323ceedf38425" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38969 } }
I20260807 19:43:14.418954 4917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:14.419049 4917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:14.419202 4917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:14.421872 4917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d2baf1a8014a3e86e323ceedf38425" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38969 } }
I20260807 19:43:14.422309 4917 leader_election.cc:302] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28d2baf1a8014a3e86e323ceedf38425; no voters:
I20260807 19:43:14.422814 4917 leader_election.cc:288] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:14.422962 4927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:14.423710 4927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [term 1 LEADER]: Becoming Leader. State: Replica: 28d2baf1a8014a3e86e323ceedf38425, State: Running, Role: LEADER
I20260807 19:43:14.424227 4927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d2baf1a8014a3e86e323ceedf38425" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38969 } }
I20260807 19:43:14.424973 4917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:14.426858 4929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28d2baf1a8014a3e86e323ceedf38425. Latest consensus state: current_term: 1 leader_uuid: "28d2baf1a8014a3e86e323ceedf38425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d2baf1a8014a3e86e323ceedf38425" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38969 } } }
I20260807 19:43:14.427237 4929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:14.428820 4932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:14.428795 4928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28d2baf1a8014a3e86e323ceedf38425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d2baf1a8014a3e86e323ceedf38425" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38969 } } }
I20260807 19:43:14.429139 4928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:14.434305 4932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:14.442360 4932 catalog_manager.cc:1347] Generated new cluster ID: a08f1cb262244a2397269fad20fe0fc0
I20260807 19:43:14.442472 4932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:14.459652 4932 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:14.460968 4932 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:14.477959 4932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28d2baf1a8014a3e86e323ceedf38425: Generated new TSK 0
I20260807 19:43:14.478801 4932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:43:14.486593 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4848
W20260807 19:43:14.487664 4912 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46985
W20260807 19:43:14.488348 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4848: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:43:14.488502 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 4848
2026-08-07T19:43:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2031 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:14 dist-test-slave-07c2 krb5kdc[4948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:14 dist-test-slave-07c2 krb5kdc[4948](info): set up 2 sockets
Aug 07 19:43:14 dist-test-slave-07c2 krb5kdc[4948](info): commencing operation
krb5kdc: starting...
W20260807 19:43:15.646652 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:43:15 dist-test-slave-07c2 krb5kdc[4948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:43:15.979041 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:43353
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:43353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:16.325210 4961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:16.325485 4961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:16.325650 4961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:16.332798 4961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:16.332906 4961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:16.333007 4961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:16.333113 4961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:16.333197 4961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:16.342887 4961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:43353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:43353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:16.344189 4961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:16.345497 4961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:16.357124 4967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:16.357226 4966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:16.357275 4969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:16.358280 4961 server_base.cc:1034] running on GCE node
I20260807 19:43:16.359155 4961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:16.360836 4961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:16.362082 4961 hybrid_clock.cc:648] HybridClock initialized: now 1786131796362011 us; error 58 us; skew 500 ppm
I20260807 19:43:16.362424 4961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:16 dist-test-slave-07c2 krb5kdc[4948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131796, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:16.370354 4961 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:43:16.372036 4961 webserver.cc:460] Webserver started at http://127.14.16.126:45243/ using document root <none> and password file <none>
I20260807 19:43:16.372735 4961 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:16.372884 4961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:16.373205 4961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:16.376734 4961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "5d701b71614e42daacf16dc9fc1a086a"
format_stamp: "Formatted at 2026-08-07 19:43:16 on dist-test-slave-07c2"
server_key: "838dd071aac90accf5577c170d9430e2"
server_key_iv: "7b4c4277cc21fe33259533a87c26429b"
server_key_version: "encryptionkey@0"
I20260807 19:43:16.377496 4961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "5d701b71614e42daacf16dc9fc1a086a"
format_stamp: "Formatted at 2026-08-07 19:43:16 on dist-test-slave-07c2"
server_key: "838dd071aac90accf5577c170d9430e2"
server_key_iv: "7b4c4277cc21fe33259533a87c26429b"
server_key_version: "encryptionkey@0"
I20260807 19:43:16.382679 4961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:43:16.386328 4976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:16.387732 4961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:43:16.387929 4961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "5d701b71614e42daacf16dc9fc1a086a"
format_stamp: "Formatted at 2026-08-07 19:43:16 on dist-test-slave-07c2"
server_key: "838dd071aac90accf5577c170d9430e2"
server_key_iv: "7b4c4277cc21fe33259533a87c26429b"
server_key_version: "encryptionkey@0"
I20260807 19:43:16.388149 4961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:16.394161 4961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:16.398399 4961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:16.398739 4961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:16.418104 4961 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:43353
I20260807 19:43:16.418154 5029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:43353 every 8 connection(s)
I20260807 19:43:16.419636 4961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:16.421793 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 4961
I20260807 19:43:16.421999 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:43:16.422575 14401 external_mini_cluster.cc:1428] Setting key a9a7fa5b80e320e6df7d563d27be1ac8
I20260807 19:43:16.424010 5030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:16.430123 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:16.424429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59059 (local address 127.14.16.126:43353)
0807 19:43:16.425248 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:16.425259 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:16.425298 (+ 39us) server_negotiation.cc:407] Connection header received
0807 19:43:16.426839 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:16.426860 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:16.427175 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:16.427572 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:16.428632 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:16.428637 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:16.428647 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:16.428659 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:16.428723 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:16.428745 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:16.428911 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:16.429046 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 19:43:16.429229 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":369,"threads_started":1}
I20260807 19:43:16.433910 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:43:16.437158 5030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a: Bootstrap starting.
I20260807 19:43:16.440891 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:16.438371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57481 (local address 127.14.16.126:43353)
0807 19:43:16.438583 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:16.438589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:16.438967 (+ 378us) server_negotiation.cc:407] Connection header received
0807 19:43:16.439137 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:16.439140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:16.439184 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:16.439343 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:16.440257 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:16.440260 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:16.440264 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:16.440277 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:16.440313 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:16.440327 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:16.440412 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:16.440560 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 19:43:16.440601 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:43:16.440929 5030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:16.442067 5030 log.cc:824] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:16.444648 5030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a: No bootstrap required, opened a new log
I20260807 19:43:16.452075 5030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d701b71614e42daacf16dc9fc1a086a" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43353 } }
I20260807 19:43:16.452469 5030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:16.452585 5030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d701b71614e42daacf16dc9fc1a086a, State: Initialized, Role: FOLLOWER
I20260807 19:43:16.453147 5030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d701b71614e42daacf16dc9fc1a086a" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43353 } }
I20260807 19:43:16.453392 5030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:16.453548 5030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:16.453730 5030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:16.456059 5030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d701b71614e42daacf16dc9fc1a086a" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43353 } }
I20260807 19:43:16.456555 5030 leader_election.cc:302] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d701b71614e42daacf16dc9fc1a086a; no voters:
I20260807 19:43:16.457351 5030 leader_election.cc:288] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:16.457461 5040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:16.458132 5040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [term 1 LEADER]: Becoming Leader. State: Replica: 5d701b71614e42daacf16dc9fc1a086a, State: Running, Role: LEADER
I20260807 19:43:16.458745 5040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d701b71614e42daacf16dc9fc1a086a" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43353 } }
I20260807 19:43:16.459720 5030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:16.462013 5041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d701b71614e42daacf16dc9fc1a086a. Latest consensus state: current_term: 1 leader_uuid: "5d701b71614e42daacf16dc9fc1a086a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d701b71614e42daacf16dc9fc1a086a" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43353 } } }
I20260807 19:43:16.461944 5042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d701b71614e42daacf16dc9fc1a086a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d701b71614e42daacf16dc9fc1a086a" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 43353 } } }
I20260807 19:43:16.462512 5041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:16.462527 5042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:16.464924 5048 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:16.469322 5048 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:16.477947 5048 catalog_manager.cc:1347] Generated new cluster ID: 6456a8d49c5f491295affe0770fb6873
I20260807 19:43:16.478072 5048 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:16.491926 5048 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:16.492416 5048 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:16.514214 5048 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d701b71614e42daacf16dc9fc1a086a: Generated new TSK 0
I20260807 19:43:16.515352 5048 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:43:16.530433 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4961
W20260807 19:43:16.531847 5025 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59059
W20260807 19:43:16.532624 14401 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 4961: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:43:16.532814 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 4961
2026-08-07T19:43:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2044 ms)
[----------] 18 tests from AuthTokenIssuingTest (37337 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:16 dist-test-slave-07c2 krb5kdc[5062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:16 dist-test-slave-07c2 krb5kdc[5062](info): set up 2 sockets
Aug 07 19:43:16 dist-test-slave-07c2 krb5kdc[5062](info): commencing operation
krb5kdc: starting...
W20260807 19:43:17.691033 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:43:17 dist-test-slave-07c2 krb5kdc[5062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:43:18.022354 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:38815
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:38815,127.14.16.125:35117,127.14.16.124:38583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:18.366707 5076 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:43:18.367240 5076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:18.367383 5076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:18.367484 5076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:18.375042 5076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:18.375123 5076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:18.375192 5076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:18.375236 5076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:18.375344 5076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:18.385164 5076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:38815,127.14.16.125:35117,127.14.16.124:38583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:38815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:18.386348 5076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:18.387578 5076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:18.399497 5081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:18.399541 5084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:18.399578 5082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:18.400225 5076 server_base.cc:1034] running on GCE node
I20260807 19:43:18.400969 5076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:18.402310 5076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:18.403551 5076 hybrid_clock.cc:648] HybridClock initialized: now 1786131798403481 us; error 46 us; skew 500 ppm
I20260807 19:43:18.403878 5076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:18 dist-test-slave-07c2 krb5kdc[5062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:18.406885 5076 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:43:18.408331 5076 webserver.cc:460] Webserver started at http://127.14.16.126:33779/ using document root <none> and password file <none>
I20260807 19:43:18.409003 5076 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:18.409137 5076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:18.409454 5076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:18.412416 5076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "5ade4451812e44a8b0309298a0c9886d"
format_stamp: "Formatted at 2026-08-07 19:43:18 on dist-test-slave-07c2"
server_key: "9ea4cdca9b0844cda245cc26f5468380"
server_key_iv: "ccc0c8fc3ee6981815d271f243f992e3"
server_key_version: "encryptionkey@0"
I20260807 19:43:18.413265 5076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "5ade4451812e44a8b0309298a0c9886d"
format_stamp: "Formatted at 2026-08-07 19:43:18 on dist-test-slave-07c2"
server_key: "9ea4cdca9b0844cda245cc26f5468380"
server_key_iv: "ccc0c8fc3ee6981815d271f243f992e3"
server_key_version: "encryptionkey@0"
I20260807 19:43:18.418284 5076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:43:18.421728 5091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:18.422998 5076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:43:18.423200 5076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "5ade4451812e44a8b0309298a0c9886d"
format_stamp: "Formatted at 2026-08-07 19:43:18 on dist-test-slave-07c2"
server_key: "9ea4cdca9b0844cda245cc26f5468380"
server_key_iv: "ccc0c8fc3ee6981815d271f243f992e3"
server_key_version: "encryptionkey@0"
I20260807 19:43:18.423480 5076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:18.441609 5076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:18.445487 5076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:18.445761 5076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:18.464890 5076 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:38815
I20260807 19:43:18.464938 5144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:38815 every 8 connection(s)
I20260807 19:43:18.466329 5076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:18.470106 5145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:18.475668 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 5076
I20260807 19:43:18.475910 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:43:18.476585 14401 external_mini_cluster.cc:1428] Setting key b48ee7e0b1226ee7886fe60cdf6ca9aa
I20260807 19:43:18.477101 5145 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:43:18 dist-test-slave-07c2 krb5kdc[5062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:43:18.501921 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.480522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:38815 (local address 127.0.0.1:47678)
0807 19:43:18.481671 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:18.481711 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:18.483609 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:18.484512 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:18.484519 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:18.485259 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:18.485618 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:18.485636 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:18.487259 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:18.487264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:18.488549 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:18.488563 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:18.489497 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:18.489504 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:18.489696 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:18.490387 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:18.490410 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:18.492905 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:18.497645 (+ 4740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:18.497659 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:18.497669 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:18.498077 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:18.498605 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:18.498609 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:18.498611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:18.498781 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:18.499944 (+ 1163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:18.499950 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:18.500173 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 19:43:18.500397 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 19:43:18.500826 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"thread_start_us":733,"threads_started":1}
I20260807 19:43:18.502023 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.480510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47678 (local address 127.14.16.126:38815)
0807 19:43:18.481392 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:18.481413 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:18.481838 (+ 425us) server_negotiation.cc:407] Connection header received
0807 19:43:18.484023 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:18.484029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:18.484275 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:18.484494 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:18.485845 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:18.487085 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:18.488693 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:18.489260 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:18.489461 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:18.493179 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:18.493192 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:18.493212 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:18.493272 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:18.497482 (+ 4210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:18.498255 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:18.498266 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:18.498272 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:18.498484 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:18.498909 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:18.498916 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:18.498922 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:18.499777 (+ 855us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:18.500029 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:18.500608 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 19:43:18.501038 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":604,"thread_start_us":392,"threads_started":1}
I20260807 19:43:18.505224 5145 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:43:18.506513 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.505964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35144)
0807 19:43:18.506186 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:18.506347 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 19:43:18.506830 5094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
I20260807 19:43:18.508412 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.508053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35150)
0807 19:43:18.508214 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:18.508280 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 19:43:18.508649 5145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:35117: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.14.16.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.125.keytab.
Entry for principal kudu/127.14.16.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.125.keytab.
I20260807 19:43:18.552541 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.125:35117
--webserver_interface=127.14.16.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.125.keytab
--principal=kudu/127.14.16.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:38815,127.14.16.125:35117,127.14.16.124:38583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:43:18.557981 5145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } attempt: 1
I20260807 19:43:18.559216 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.558750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35160)
0807 19:43:18.558961 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:18.559072 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 19:43:18.560253 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.559862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35164)
0807 19:43:18.560011 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:18.560086 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 19:43:18.560454 5145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:35117: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
I20260807 19:43:18.628718 5145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } attempt: 2
I20260807 19:43:18.630036 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.629552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35168)
0807 19:43:18.629781 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:18.629877 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 19:43:18.631095 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.630719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35178)
0807 19:43:18.630880 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:18.630965 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 19:43:18.631330 5145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:35117: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
I20260807 19:43:18.722571 5145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } attempt: 3
I20260807 19:43:18.723784 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.723324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35182)
0807 19:43:18.723525 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:18.723651 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:43:18.724853 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.724492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35186)
0807 19:43:18.724656 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:18.724719 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 19:43:18.725044 5145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:35117: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
I20260807 19:43:18.868299 5145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } attempt: 4
I20260807 19:43:18.869511 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.868973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35194)
0807 19:43:18.869286 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:18.869375 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260807 19:43:18.870584 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.870228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35204)
0807 19:43:18.870381 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:18.870460 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 19:43:18.870777 5145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:35117: Network error: Client connection negotiation failed: client connection to 127.14.16.125:35117: connect: Connection refused (error 111)
W20260807 19:43:18.891803 5152 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:43:18.892168 5152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:18.892293 5152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:18.892428 5152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:18.899634 5152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:18.899760 5152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:18.899888 5152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:18.899977 5152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:18.900089 5152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:18.909927 5152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:38815,127.14.16.125:35117,127.14.16.124:38583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.125.keytab
--principal=kudu/127.14.16.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.125:35117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.16.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:18.911209 5152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:18.912539 5152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:18.924400 5158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:18.924398 5160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:18.924628 5162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:18.925493 5152 server_base.cc:1034] running on GCE node
I20260807 19:43:18.926241 5152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:18.927685 5152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:18.928897 5152 hybrid_clock.cc:648] HybridClock initialized: now 1786131798928813 us; error 64 us; skew 500 ppm
I20260807 19:43:18.929232 5152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:18 dist-test-slave-07c2 krb5kdc[5062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:18.932569 5152 init.cc:377] Logged in from keytab as kudu/127.14.16.125@KRBTEST.COM (short username kudu)
I20260807 19:43:18.934043 5152 webserver.cc:460] Webserver started at http://127.14.16.125:37565/ using document root <none> and password file <none>
I20260807 19:43:18.934736 5152 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:18.934854 5152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:18.935200 5152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:18.938201 5152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/data/instance:
uuid: "696551f23fa343d0befc8bd972448020"
format_stamp: "Formatted at 2026-08-07 19:43:18 on dist-test-slave-07c2"
server_key: "930a86af9f8b275fbbfbd87ce27bca47"
server_key_iv: "85903f4a8e43c98153fc3f5a4d114111"
server_key_version: "encryptionkey@0"
I20260807 19:43:18.939004 5152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/wal/instance:
uuid: "696551f23fa343d0befc8bd972448020"
format_stamp: "Formatted at 2026-08-07 19:43:18 on dist-test-slave-07c2"
server_key: "930a86af9f8b275fbbfbd87ce27bca47"
server_key_iv: "85903f4a8e43c98153fc3f5a4d114111"
server_key_version: "encryptionkey@0"
I20260807 19:43:18.944366 5152 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:43:18.947938 5169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:18.949335 5152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:43:18.949525 5152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/wal
uuid: "696551f23fa343d0befc8bd972448020"
format_stamp: "Formatted at 2026-08-07 19:43:18 on dist-test-slave-07c2"
server_key: "930a86af9f8b275fbbfbd87ce27bca47"
server_key_iv: "85903f4a8e43c98153fc3f5a4d114111"
server_key_version: "encryptionkey@0"
I20260807 19:43:18.949729 5152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:18.968931 5152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:18.973150 5152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:18.973423 5152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:18.992175 5152 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.125:35117
I20260807 19:43:18.992192 5220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.125:35117 every 8 connection(s)
I20260807 19:43:18.993523 5152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
I20260807 19:43:18.994529 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 5152
I20260807 19:43:18.994745 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-1/wal/instance
I20260807 19:43:18.995507 14401 external_mini_cluster.cc:1428] Setting key b920ac85b5a10d7591d1f256c851e06d
I20260807 19:43:18.999774 5221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:19.008668 5221 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:43:19 dist-test-slave-07c2 krb5kdc[5062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.125@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.16.124@KRBTEST.COM; defaulting to no policy
I20260807 19:43:19.029474 5224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.011418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47682 (local address 127.14.16.126:38815)
0807 19:43:19.011919 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.011923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.012824 (+ 901us) server_negotiation.cc:407] Connection header received
0807 19:43:19.014660 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.014665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.014756 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.014878 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.016448 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.017186 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.018675 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.019112 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.019281 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.023167 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.023172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.023175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.023212 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.026082 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.026948 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.026957 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.026959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.027076 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.027646 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.027651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.027654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.028014 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.028186 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.028864 (+ 678us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.029090 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":290,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.124@KRBTEST.COM" created.
I20260807 19:43:19.030422 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.011943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:38815 (local address 127.0.0.1:47682)
0807 19:43:19.012656 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.012697 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:19.014393 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:19.015117 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:19.015125 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:19.015687 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:19.016235 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.016251 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.017362 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.017366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.018525 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.018536 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.019402 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.019410 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.019642 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.020354 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:19.020375 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:19.022916 (+ 2541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:19.026259 (+ 3343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.026267 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.026285 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.026803 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.027335 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.027339 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.027349 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.027520 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.028148 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:19.028156 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:19.028494 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 19:43:19.028770 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 19:43:19.029276 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":290,"threads_started":1}
I20260807 19:43:19.032258 5221 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:43:19 dist-test-slave-07c2 krb5kdc[5062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.125@KRBTEST.COM for kudu/127.14.16.125@KRBTEST.COM
I20260807 19:43:19.046538 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.033070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35210)
0807 19:43:19.033238 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.033263 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:19.033395 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:19.034091 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:19.034096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:19.034520 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:19.034644 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.034653 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.036053 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.036058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.036874 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.036885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.037653 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.037659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.037891 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.038474 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:19.038491 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:19.040683 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:19.043985 (+ 3302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.043993 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.043997 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.044358 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.044761 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.044766 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.044770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.044923 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.045528 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:19.045533 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:19.045673 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 19:43:19.045824 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 19:43:19.046111 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":3}
I20260807 19:43:19.046614 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.033137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35210 (local address 127.14.16.125:35117)
0807 19:43:19.033726 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.033731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.033754 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:43:19.033792 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.033797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.033947 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.034105 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.034816 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.035915 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.037005 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.037511 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.037653 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.040883 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.040893 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.040904 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.040931 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.043852 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.044506 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.044512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.044514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.044649 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.045053 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.045056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.045058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.045402 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.045626 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.045945 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.046209 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":443,"thread_start_us":285,"threads_started":1}
I20260807 19:43:19.049201 5221 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:43:19.050383 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.049936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:35882)
0807 19:43:19.050145 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.050241 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 19:43:19.050719 5170 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
I20260807 19:43:19.052352 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.052029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:35890)
0807 19:43:19.052183 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.052256 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 19:43:19.052672 5221 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:38583: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.124.keytab.
Entry for principal kudu/127.14.16.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.124.keytab.
I20260807 19:43:19.070536 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.124:38583
--webserver_interface=127.14.16.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.124.keytab
--principal=kudu/127.14.16.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:38815,127.14.16.125:35117,127.14.16.124:38583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:43:19.102010 5221 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } attempt: 1
I20260807 19:43:19.103246 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.102793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:35894)
0807 19:43:19.102984 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.103094 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 19:43:19.104338 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.103969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:35908)
0807 19:43:19.104148 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.104224 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 19:43:19.104554 5221 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:38583: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
I20260807 19:43:19.170042 5145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } attempt: 5
I20260807 19:43:19.172801 5221 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } attempt: 2
I20260807 19:43:19.173820 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.173338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:35918)
0807 19:43:19.173521 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.173623 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 19:43:19.174961 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.174559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:35924)
0807 19:43:19.174726 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.174813 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 19:43:19.175276 5221 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:38583: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Aug 07 19:43:19 dist-test-slave-07c2 krb5kdc[5062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for kudu/127.14.16.125@KRBTEST.COM
I20260807 19:43:19.185563 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.170852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35214)
0807 19:43:19.171131 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.171159 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:19.171362 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:19.171748 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:19.171754 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:19.172305 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:19.172541 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.172554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.173812 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.173819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.175119 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.175135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.176384 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.176389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.176628 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.177309 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:19.177327 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:19.179564 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:19.183360 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.183364 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.183366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.183735 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.184120 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.184123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.184137 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.184263 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.184859 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:19.184862 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:19.184992 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 19:43:19.185119 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 19:43:19.185327 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 19:43:19.185802 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.170911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35214 (local address 127.14.16.125:35117)
0807 19:43:19.171125 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.171129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.171273 (+ 144us) server_negotiation.cc:407] Connection header received
0807 19:43:19.171493 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.171497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.171561 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.171673 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.172704 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.173673 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.175333 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.176191 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.176360 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.179757 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.179764 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.179769 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.179807 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.183168 (+ 3361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.183853 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.183859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.183863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.184010 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.184380 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.184385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.184388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.184741 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.184890 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.185303 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.185529 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:43:19.186746 5145 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:43:19.187788 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.187425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:35934)
0807 19:43:19.187578 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.187656 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 19:43:19.188795 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.188430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:35946)
0807 19:43:19.188602 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.188688 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 19:43:19.189013 5145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:38583: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
I20260807 19:43:19.240286 5145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } attempt: 1
I20260807 19:43:19.241561 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.241079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:35962)
0807 19:43:19.241305 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.241398 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 19:43:19.242722 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.242353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:35964)
0807 19:43:19.242520 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.242606 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 19:43:19.242955 5145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:38583: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
I20260807 19:43:19.266556 5221 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } attempt: 3
I20260807 19:43:19.267792 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.267343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:35980)
0807 19:43:19.267527 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.267633 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:43:19.268908 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.268448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:35994)
0807 19:43:19.268709 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.268783 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260807 19:43:19.269122 5221 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:38583: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
I20260807 19:43:19.311180 5145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } attempt: 2
I20260807 19:43:19.312403 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.311899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:36008)
0807 19:43:19.312130 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.312238 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 19:43:19.313468 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.313128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:36010)
0807 19:43:19.313294 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.313357 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 19:43:19.313681 5145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:38583: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
I20260807 19:43:19.412346 5221 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } attempt: 4
I20260807 19:43:19.413503 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.412991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:36014)
0807 19:43:19.413251 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.413338 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 19:43:19.414475 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.414156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:36020)
0807 19:43:19.414300 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.414372 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 19:43:19.414688 5221 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:38583: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
W20260807 19:43:19.416908 5229 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:43:19.417253 5229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:19.417383 5229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:19.417523 5229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 19:43:19.419886 5145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } attempt: 3
I20260807 19:43:19.420951 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.420425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:36024)
0807 19:43:19.420639 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.420750 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:43:19.422053 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.421613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:36034)
0807 19:43:19.421820 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.421919 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260807 19:43:19.422281 5145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:38583: Network error: Client connection negotiation failed: client connection to 127.14.16.124:38583: connect: Connection refused (error 111)
W20260807 19:43:19.425607 5229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:19.425686 5229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:19.425835 5229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:19.425910 5229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:19.426019 5229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:19.436023 5229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:38815,127.14.16.125:35117,127.14.16.124:38583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.124.keytab
--principal=kudu/127.14.16.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.124:38583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.16.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:19.437292 5229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:19.438625 5229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:19.450429 5237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:19.450588 5235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:19.450562 5234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:19.451570 5229 server_base.cc:1034] running on GCE node
I20260807 19:43:19.452299 5229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:19.453615 5229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:19.454875 5229 hybrid_clock.cc:648] HybridClock initialized: now 1786131799454791 us; error 59 us; skew 500 ppm
I20260807 19:43:19.455227 5229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:19 dist-test-slave-07c2 krb5kdc[5062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131799, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:19.458340 5229 init.cc:377] Logged in from keytab as kudu/127.14.16.124@KRBTEST.COM (short username kudu)
I20260807 19:43:19.459872 5229 webserver.cc:460] Webserver started at http://127.14.16.124:42831/ using document root <none> and password file <none>
I20260807 19:43:19.460506 5229 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:19.460593 5229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:19.460875 5229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:19.463877 5229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/data/instance:
uuid: "1414060244dc499aa637127e18ba2330"
format_stamp: "Formatted at 2026-08-07 19:43:19 on dist-test-slave-07c2"
server_key: "c4e1addea632560672207ff0a5674ac2"
server_key_iv: "2279835bb5cc1be4510b839e94518d04"
server_key_version: "encryptionkey@0"
I20260807 19:43:19.464892 5229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/wal/instance:
uuid: "1414060244dc499aa637127e18ba2330"
format_stamp: "Formatted at 2026-08-07 19:43:19 on dist-test-slave-07c2"
server_key: "c4e1addea632560672207ff0a5674ac2"
server_key_iv: "2279835bb5cc1be4510b839e94518d04"
server_key_version: "encryptionkey@0"
I20260807 19:43:19.471464 5229 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:43:19.475937 5245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:19.477283 5229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:43:19.477468 5229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/wal
uuid: "1414060244dc499aa637127e18ba2330"
format_stamp: "Formatted at 2026-08-07 19:43:19 on dist-test-slave-07c2"
server_key: "c4e1addea632560672207ff0a5674ac2"
server_key_iv: "2279835bb5cc1be4510b839e94518d04"
server_key_version: "encryptionkey@0"
I20260807 19:43:19.477707 5229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:19.499318 5229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:19.503779 5229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:19.504067 5229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:19.523957 5229 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.124:38583
I20260807 19:43:19.523976 5298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.124:38583 every 8 connection(s)
I20260807 19:43:19.525350 5229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
I20260807 19:43:19.529479 5299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:19.533363 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 5229
I20260807 19:43:19.533573 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131435615921-14401-0/minicluster-data/master-2/wal/instance
I20260807 19:43:19.534128 14401 external_mini_cluster.cc:1428] Setting key eecb87f48c187c2c580a55da8f4d60e8
I20260807 19:43:19.537559 5299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:43:19 dist-test-slave-07c2 krb5kdc[5062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.124@KRBTEST.COM
Aug 07 19:43:19 dist-test-slave-07c2 krb5kdc[5062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131799, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.124@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:43:19.556954 5303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.541316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47694 (local address 127.14.16.126:38815)
0807 19:43:19.541877 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.541885 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.542459 (+ 574us) server_negotiation.cc:407] Connection header received
0807 19:43:19.542649 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.542655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.542748 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.542873 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.544115 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.544815 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.546410 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.546817 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.546957 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.550819 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.550825 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.550828 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.550864 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.553799 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.554711 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.554718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.554725 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.554887 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.555281 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.555326 (+ 45us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.555331 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.555612 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.555785 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.556406 (+ 621us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.556643 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":286,"threads_started":1}
I20260807 19:43:19.558385 5302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.536176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36038 (local address 127.14.16.124:38583)
0807 19:43:19.536904 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.536916 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.536956 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:43:19.538594 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.538616 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.539041 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.539508 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.541220 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.543063 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.544404 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.545084 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.545298 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.549826 (+ 4528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.549838 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.549881 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.549928 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.553206 (+ 3278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.554163 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.554173 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.554177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.554372 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.554803 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.554807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.554809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.555566 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.555854 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.556254 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.556957 (+ 703us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":337,"server-negotiator.queue_time_us":526,"thread_start_us":322,"threads_started":1}
I20260807 19:43:19.558404 5304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.541307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:38815 (local address 127.0.0.1:47694)
0807 19:43:19.542305 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.542354 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:19.542509 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:19.542971 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:19.542977 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:19.543712 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:19.543956 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.543965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.544980 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.544995 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.546251 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.546263 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.547056 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.547064 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.547227 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.547910 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:19.547936 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:19.550574 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:19.553932 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.553942 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.553952 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.554595 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.554999 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.555003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.555006 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.555173 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.555777 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:19.555785 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:19.555988 (+ 203us) client_negotiation.cc:769] Sending connection context
0807 19:43:19.556274 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 19:43:19.556863 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":4,"thread_start_us":545,"threads_started":1}
I20260807 19:43:19.560252 5299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:43:19 dist-test-slave-07c2 krb5kdc[5062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131799, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.124@KRBTEST.COM for kudu/127.14.16.125@KRBTEST.COM
Aug 07 19:43:19 dist-test-slave-07c2 krb5kdc[5062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:43:19.577497 5304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.561210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:35117 (local address 127.0.0.1:35222)
0807 19:43:19.561488 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.561506 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:19.561658 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:19.562149 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:19.562152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:19.562497 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:19.562651 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.562660 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.564090 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.564096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.565455 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.565466 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.566409 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.566413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.566562 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.567194 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:19.567212 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:19.569550 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:19.573966 (+ 4416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.573971 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.573975 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.574661 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.575245 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.575254 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.575261 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.575603 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.576312 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:19.576319 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:19.576658 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 19:43:19.576842 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 19:43:19.577152 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260807 19:43:19.578042 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.561248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35222 (local address 127.14.16.125:35117)
0807 19:43:19.561457 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.561463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.561589 (+ 126us) server_negotiation.cc:407] Connection header received
0807 19:43:19.561782 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.561789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.561874 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.561970 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.562799 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.563946 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.565589 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.566270 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.566407 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.569757 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.569768 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.569774 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.569825 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.573815 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.574799 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.574809 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.574846 (+ 37us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.575056 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.575774 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.575779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.575782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.576173 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.576340 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.577009 (+ 669us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.577460 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:43:19.579126 5299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:43:19.581362 5303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.561941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47704 (local address 127.14.16.126:38815)
0807 19:43:19.562175 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.562181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.562200 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:43:19.562322 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.562327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.562406 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.562511 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.564090 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.565159 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.566895 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.567412 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.567552 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.572748 (+ 5196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.572758 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.572766 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.572815 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.576516 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.577403 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.577410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.577416 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.577866 (+ 450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.578600 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.578605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.578608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.579030 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.580563 (+ 1533us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.580734 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.580985 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
Aug 07 19:43:19 dist-test-slave-07c2 krb5kdc[5062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131799, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.124@KRBTEST.COM for kudu/127.14.16.124@KRBTEST.COM
Aug 07 19:43:19 dist-test-slave-07c2 krb5kdc[5062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.125@KRBTEST.COM
I20260807 19:43:19.594247 5304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.579952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:36044)
0807 19:43:19.580150 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.580169 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:19.580275 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:19.580993 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:19.580997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:19.581552 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:19.581726 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.581735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.583470 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.583474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.584415 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.584425 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.585137 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.585143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.585382 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.585974 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:19.585991 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:19.588006 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:19.591680 (+ 3674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.591685 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.591689 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.592233 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.592595 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.592603 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.592606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.592756 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.593369 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:19.593375 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:19.593594 (+ 219us) client_negotiation.cc:769] Sending connection context
0807 19:43:19.593733 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 19:43:19.593945 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":20}
I20260807 19:43:19.594551 5302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.580477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36044 (local address 127.14.16.124:38583)
0807 19:43:19.580679 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.580685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.580704 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:43:19.580751 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.580756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.580847 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.580963 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.582508 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.583338 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.584557 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.584989 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.585118 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.588186 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.588193 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.588199 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.588239 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.591516 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.592352 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.592355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.592357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.592471 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.592872 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.592878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.592883 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.593241 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.593388 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.593868 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.594106 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":62}
I20260807 19:43:19.599443 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.583101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35224 (local address 127.14.16.125:35117)
0807 19:43:19.583260 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.583266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.583311 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:43:19.583448 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.583454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.583528 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.583619 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.584788 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.585493 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.586658 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.587101 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.587234 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.592660 (+ 5426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.592664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.592668 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.592699 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.596611 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.597281 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.597288 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.597291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.597450 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.597883 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.597888 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.597891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.598259 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.598515 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.598808 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.599041 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:43:19.599529 5145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } attempt: 4
I20260807 19:43:19.600637 5299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330: Bootstrap starting.
I20260807 19:43:19.604699 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:43:19.604873 5299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:19.606251 5299 log.cc:824] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:43:19 dist-test-slave-07c2 krb5kdc[5062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for kudu/127.14.16.124@KRBTEST.COM
I20260807 19:43:19.609743 5299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330: No bootstrap required, opened a new log
I20260807 19:43:19.608171 5119 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5ade4451812e44a8b0309298a0c9886d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47704
I20260807 19:43:19.614868 5302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.600298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36050 (local address 127.14.16.124:38583)
0807 19:43:19.600583 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.600590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.600609 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:43:19.600860 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.600866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.600945 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.601060 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.601818 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.602841 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.604685 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.605329 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.605472 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.609139 (+ 3667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.609145 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.609150 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.609190 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.612144 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.612850 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.612857 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.612861 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.613001 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.613382 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.613388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.613392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.613709 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.613883 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.614270 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.614487 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260807 19:43:19.614864 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.600210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:36050)
0807 19:43:19.600493 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.600514 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:19.600642 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:19.601103 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:19.601108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:19.601505 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:19.601673 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.601682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.603005 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.603013 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.604526 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.604539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.605482 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.605487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.605713 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.606341 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:19.606387 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:19.608954 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:19.612350 (+ 3396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.612357 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.612366 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.612731 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.613117 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.613124 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.613127 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.613273 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.613842 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:19.613847 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:19.614015 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 19:43:19.614170 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 19:43:19.614458 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260807 19:43:19.619493 5145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d: Bootstrap starting.
I20260807 19:43:19.620584 5299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } }
I20260807 19:43:19.620934 5299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1414060244dc499aa637127e18ba2330, State: Initialized, Role: FOLLOWER
I20260807 19:43:19.621438 5299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } }
I20260807 19:43:19.622440 5306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } } }
I20260807 19:43:19.622839 5306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:19.623317 5145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:19.623847 5299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:19.624428 5145 log.cc:824] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:19.627962 5145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d: No bootstrap required, opened a new log
W20260807 19:43:19.633945 5318 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:43:19.634071 5318 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:43:19.635999 5145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } }
I20260807 19:43:19.636406 5145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ade4451812e44a8b0309298a0c9886d, State: Initialized, Role: FOLLOWER
I20260807 19:43:19.636895 5145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } }
I20260807 19:43:19.643860 5321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } } }
I20260807 19:43:19.644243 5321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:19.631865 5229 master_runner.cc:186] Error getting master registration for 127.14.16.126:38815: OK, instance_id { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" instance_seqno: 1786131798459405 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:43:19.645047 5229 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:43:19.645906 5145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:19.646981 5119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:43:19.647192 5119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:19.650767 5119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } }
I20260807 19:43:19.654417 5273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ade4451812e44a8b0309298a0c9886d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1414060244dc499aa637127e18ba2330"
I20260807 19:43:19.654884 5273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:19.658429 5273 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ade4451812e44a8b0309298a0c9886d in term 1.
I20260807 19:43:19.660169 5092 leader_election.cc:302] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1414060244dc499aa637127e18ba2330, 5ade4451812e44a8b0309298a0c9886d; no voters:
W20260807 19:43:19.660574 5333 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:43:19.660694 5333 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:43:19.660684 5321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:19.663465 5321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [term 1 LEADER]: Becoming Leader. State: Replica: 5ade4451812e44a8b0309298a0c9886d, State: Running, Role: LEADER
I20260807 19:43:19.664083 5321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } }
I20260807 19:43:19.668109 5119 leader_election.cc:288] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [CANDIDATE]: Term 1 election: Requested vote from peers 696551f23fa343d0befc8bd972448020 (127.14.16.125:35117), 1414060244dc499aa637127e18ba2330 (127.14.16.124:38583)
I20260807 19:43:19.662675 5076 master_runner.cc:186] Error getting master registration for 127.14.16.125:35117: OK, instance_id { permanent_uuid: "696551f23fa343d0befc8bd972448020" instance_seqno: 1786131798986842 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:43:19.674479 5076 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:43:19.684957 5335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ade4451812e44a8b0309298a0c9886d. Latest consensus state: current_term: 1 leader_uuid: "5ade4451812e44a8b0309298a0c9886d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } } }
I20260807 19:43:19.685388 5335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:19.686316 5339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:19.688903 5339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 19:43:19.699536 5097 tablet.cc:2369] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:43:19.713820 5221 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } attempt: 5
Aug 07 19:43:19 dist-test-slave-07c2 krb5kdc[5062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.125@KRBTEST.COM for kudu/127.14.16.124@KRBTEST.COM
I20260807 19:43:19.737142 5273 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [term 1 FOLLOWER]: Refusing update from remote peer 5ade4451812e44a8b0309298a0c9886d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:19.738237 5335 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:19.740671 5302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.702090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36052 (local address 127.14.16.124:38583)
0807 19:43:19.702317 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.702324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.706967 (+ 4643us) server_negotiation.cc:407] Connection header received
0807 19:43:19.707335 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.707340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.707474 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.707588 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.711446 (+ 3858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.712636 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.725240 (+ 12604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.725913 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.726020 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.732610 (+ 6590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.732616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.732622 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.732663 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.736663 (+ 4000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.737574 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.737582 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.737591 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.737774 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.738255 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.738260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.738264 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.738610 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.738877 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.739941 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.740206 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:43:19.746393 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.714473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:38583 (local address 127.0.0.1:36062)
0807 19:43:19.714813 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.714833 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:19.714940 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:19.716173 (+ 1233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:19.716178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:19.716726 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:19.716945 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.716956 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.720731 (+ 3775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.720738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.722176 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.722195 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.723283 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.723305 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.723532 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.724576 (+ 1044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:19.724599 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:19.727982 (+ 3383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:19.742902 (+ 14920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.742909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.742915 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.743476 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.744192 (+ 716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.744197 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.744201 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.744354 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.745360 (+ 1006us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:19.745365 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:19.745526 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 19:43:19.745687 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 19:43:19.745991 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260807 19:43:19.752337 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.703302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35226 (local address 127.14.16.125:35117)
0807 19:43:19.703479 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.703485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.711578 (+ 8093us) server_negotiation.cc:407] Connection header received
0807 19:43:19.716172 (+ 4594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.716178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.716246 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.716314 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.724102 (+ 7788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.725233 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.730288 (+ 5055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.730926 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.731022 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.737357 (+ 6335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.737363 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.737368 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.737406 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.741096 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.746742 (+ 5646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.746752 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.746757 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.746995 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.747872 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.747879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.747883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.748324 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.748630 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.751369 (+ 2739us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.751725 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 19:43:19.753973 5303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.705740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47708 (local address 127.14.16.126:38815)
0807 19:43:19.705931 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.705938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.715611 (+ 9673us) server_negotiation.cc:407] Connection header received
0807 19:43:19.718745 (+ 3134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.718753 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.718845 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.718931 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.727222 (+ 8291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.728350 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.731873 (+ 3523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.732565 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.732666 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.742192 (+ 9526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.742200 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.742205 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.742250 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.745899 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.749218 (+ 3319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.749228 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.749234 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.749406 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.749932 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.749941 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.749947 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.750396 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.750719 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.753228 (+ 2509us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.753627 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:43:19.755012 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.714592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.14.16.124:38583)
0807 19:43:19.715403 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.715409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.715434 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:43:19.715487 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.715491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.715580 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.717123 (+ 1543us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.717437 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.718782 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.722446 (+ 3664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.723102 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.723863 (+ 761us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.728389 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.728397 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.728403 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.728468 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.742682 (+ 14214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.743689 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.743720 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.743723 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.744002 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.744540 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.744545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.744549 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.745156 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.747117 (+ 1961us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.753509 (+ 6392us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.753984 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":334,"threads_started":1}
W20260807 19:43:19.756098 5251 tablet.cc:2369] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:43:19.762950 5221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020: Bootstrap starting.
I20260807 19:43:19.767187 5221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:19.769140 5221 log.cc:824] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:19.773633 5306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ade4451812e44a8b0309298a0c9886d. Latest consensus state: current_term: 1 leader_uuid: "5ade4451812e44a8b0309298a0c9886d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } } }
I20260807 19:43:19.773940 5306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:19.781468 5221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020: No bootstrap required, opened a new log
I20260807 19:43:19.788904 5342 mvcc.cc:205] Tried to move back new op lower bound from 7315995851680038912 to 7315995851429412864. Current Snapshot: MvccSnapshot[applied={T|T < 7315995851680038912}]
I20260807 19:43:19.791461 5306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5ade4451812e44a8b0309298a0c9886d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } } }
I20260807 19:43:19.791805 5306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:19.799615 5335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5ade4451812e44a8b0309298a0c9886d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } } }
I20260807 19:43:19.800000 5335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:19.804623 5339 catalog_manager.cc:1347] Generated new cluster ID: a63a1d4a9a1845d98224bae088b8e3ba
I20260807 19:43:19.804726 5339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:19.807605 5221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } }
I20260807 19:43:19.808099 5221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696551f23fa343d0befc8bd972448020, State: Initialized, Role: FOLLOWER
I20260807 19:43:19.811810 5221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } }
I20260807 19:43:19.813199 5354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } } }
I20260807 19:43:19.814519 5354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:19.815814 5196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ade4451812e44a8b0309298a0c9886d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "696551f23fa343d0befc8bd972448020"
I20260807 19:43:19.816201 5195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:19.816794 5221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:19.819922 5195 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [term 1 FOLLOWER]: Refusing update from remote peer 5ade4451812e44a8b0309298a0c9886d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:19.823698 5196 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ade4451812e44a8b0309298a0c9886d in term 1.
I20260807 19:43:19.826313 5335 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [LEADER]: Connected to new peer: Peer: permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:19.851442 5339 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:19.853484 5339 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:19.879146 5339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d: Generated new TSK 0
I20260807 19:43:19.880487 5339 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:43:19.894790 5354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ade4451812e44a8b0309298a0c9886d. Latest consensus state: current_term: 1 leader_uuid: "5ade4451812e44a8b0309298a0c9886d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } } }
I20260807 19:43:19.895180 5354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:19.895663 5360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5ade4451812e44a8b0309298a0c9886d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } } }
I20260807 19:43:19.896030 5360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:19.897837 5335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5ade4451812e44a8b0309298a0c9886d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ade4451812e44a8b0309298a0c9886d" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 38815 } } peers { permanent_uuid: "696551f23fa343d0befc8bd972448020" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 35117 } } peers { permanent_uuid: "1414060244dc499aa637127e18ba2330" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 38583 } } }
I20260807 19:43:19.898087 5335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ade4451812e44a8b0309298a0c9886d [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:19.914153 5371 catalog_manager.cc:1259] Loaded cluster ID: a63a1d4a9a1845d98224bae088b8e3ba
I20260807 19:43:19.914268 5371 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020: loading cluster ID for follower catalog manager: success
I20260807 19:43:19.917917 5371 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020: acquiring CA information for follower catalog manager: success
I20260807 19:43:19.920367 5371 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 696551f23fa343d0befc8bd972448020: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 19:43:20.034545 14401 mini_kdc.cc:348] Time spent kdestroy: real 0.134s user 0.005s sys 0.024s
W20260807 19:43:20.096467 5303 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:20.084140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47718 (local address 127.14.16.126:38815)
0807 19:43:20.084605 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.084614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.092324 (+ 7710us) server_negotiation.cc:407] Connection header received
0807 19:43:20.093480 (+ 1156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.093490 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.093605 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.093700 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.096177 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47718: BlockingRecv error: recv got EOF from 127.0.0.1:47718 (error 108)
Metrics: {"server-negotiator.queue_time_us":298}
W20260807 19:43:20.099668 5226 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:20.084364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35230 (local address 127.14.16.125:35117)
0807 19:43:20.084827 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.084835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.097422 (+ 12587us) server_negotiation.cc:407] Connection header received
0807 19:43:20.098109 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.098115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.098218 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.098290 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.099297 (+ 1007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35230: BlockingRecv error: recv got EOF from 127.0.0.1:35230 (error 108)
Metrics: {"server-negotiator.queue_time_us":285}
W20260807 19:43:20.103359 5346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:20.083267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36076 (local address 127.14.16.124:38583)
0807 19:43:20.083515 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.083521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.100835 (+ 17314us) server_negotiation.cc:407] Connection header received
0807 19:43:20.101570 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.101580 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.101697 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.101788 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.103070 (+ 1282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36076: BlockingRecv error: recv got EOF from 127.0.0.1:36076 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 19:43:20.152895 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.136966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35240 (local address 127.14.16.125:35117)
0807 19:43:20.137179 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.137189 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.144839 (+ 7650us) server_negotiation.cc:407] Connection header received
0807 19:43:20.145381 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.145388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.145601 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.145745 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.146466 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.147632 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.149762 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.150450 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.150551 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.151373 (+ 822us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.152048 (+ 675us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.152182 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.152430 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.152534 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 19:43:20.200088 5346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:20.188259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36080 (local address 127.14.16.124:38583)
0807 19:43:20.188451 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.188458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.197940 (+ 9482us) server_negotiation.cc:407] Connection header received
0807 19:43:20.198312 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.198319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.198453 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.198535 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.199812 (+ 1277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36080: BlockingRecv error: recv got EOF from 127.0.0.1:36080 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:43:20.251791 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.236959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35248 (local address 127.14.16.125:35117)
0807 19:43:20.237176 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.237183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.244810 (+ 7627us) server_negotiation.cc:407] Connection header received
0807 19:43:20.245332 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.245338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.245437 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.245504 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.246217 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.247458 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.249520 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.250168 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.250263 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.250890 (+ 627us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.251062 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.251135 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.251389 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.251472 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 19:43:20.302536 5346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:20.289604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36082 (local address 127.14.16.124:38583)
0807 19:43:20.289804 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.289812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.300468 (+ 10656us) server_negotiation.cc:407] Connection header received
0807 19:43:20.300979 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.300986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.301084 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.301173 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.302281 (+ 1108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36082: BlockingRecv error: recv got EOF from 127.0.0.1:36082 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:43:20.322525 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.315616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35264 (local address 127.14.16.125:35117)
0807 19:43:20.315796 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.315802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.316363 (+ 561us) server_negotiation.cc:407] Connection header received
0807 19:43:20.316596 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.316602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.316708 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.316827 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.317321 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.318549 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.320433 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.321123 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.321263 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.321701 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.321859 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.321951 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.322105 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.322195 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 19:43:20.333321 5346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:20.330241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.14.16.124:38583)
0807 19:43:20.330447 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.330453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.331776 (+ 1323us) server_negotiation.cc:407] Connection header received
0807 19:43:20.331966 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.331972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.332071 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.332196 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.333062 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36088: BlockingRecv error: recv got EOF from 127.0.0.1:36088 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:43:20.352169 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.345150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35270 (local address 127.14.16.125:35117)
0807 19:43:20.345381 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.345387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.345909 (+ 522us) server_negotiation.cc:407] Connection header received
0807 19:43:20.346115 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.346121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.346213 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.346303 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.346746 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.347991 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.349917 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.350673 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.350799 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.351344 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.351489 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.351588 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.351784 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.351866 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
W20260807 19:43:20.365355 5346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:20.359433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36094 (local address 127.14.16.124:38583)
0807 19:43:20.359674 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.359682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.363793 (+ 4111us) server_negotiation.cc:407] Connection header received
0807 19:43:20.363938 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.363943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.364048 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.364165 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.365093 (+ 928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36094: BlockingRecv error: recv got EOF from 127.0.0.1:36094 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
I20260807 19:43:20.385403 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.377959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.14.16.125:35117)
0807 19:43:20.378168 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.378175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.378707 (+ 532us) server_negotiation.cc:407] Connection header received
0807 19:43:20.378911 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.378917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.379007 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.379098 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.379769 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.380993 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.383044 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.383891 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.384027 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.384520 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.384673 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.384769 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.384937 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.385024 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 19:43:20.400269 5346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:20.397943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36108 (local address 127.14.16.124:38583)
0807 19:43:20.398204 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.398211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.398516 (+ 305us) server_negotiation.cc:407] Connection header received
0807 19:43:20.398811 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.398816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.398891 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.398975 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.400001 (+ 1026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36108: BlockingRecv error: recv got EOF from 127.0.0.1:36108 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:43:20.427064 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.420441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35276 (local address 127.14.16.125:35117)
0807 19:43:20.420650 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.420656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.421178 (+ 522us) server_negotiation.cc:407] Connection header received
0807 19:43:20.421344 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.421351 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.421457 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.421588 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.422055 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.423352 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.424926 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.425603 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.425749 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.426176 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.426323 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.426421 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.426669 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.426772 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 19:43:20.436393 5346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:20.433999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36120 (local address 127.14.16.124:38583)
0807 19:43:20.434282 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.434291 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.434880 (+ 589us) server_negotiation.cc:407] Connection header received
0807 19:43:20.435118 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.435124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.435211 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.435336 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.436129 (+ 793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36120: BlockingRecv error: recv got EOF from 127.0.0.1:36120 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 19:43:20.487804 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.480928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35284 (local address 127.14.16.125:35117)
0807 19:43:20.481176 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.481183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.481694 (+ 511us) server_negotiation.cc:407] Connection header received
0807 19:43:20.481914 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.481920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.482021 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.482153 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.482566 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.483689 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.485646 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.486343 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.486457 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.486972 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.487119 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.487202 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.487381 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.487460 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 19:43:20.496297 5346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:20.494219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36126 (local address 127.14.16.124:38583)
0807 19:43:20.494426 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.494432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.494760 (+ 328us) server_negotiation.cc:407] Connection header received
0807 19:43:20.494960 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.494964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.495034 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.495116 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.496025 (+ 909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36126: BlockingRecv error: recv got EOF from 127.0.0.1:36126 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 19:43:20.569530 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.564118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35288 (local address 127.14.16.125:35117)
0807 19:43:20.564372 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.564379 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.564670 (+ 291us) server_negotiation.cc:407] Connection header received
0807 19:43:20.564856 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.564860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.564929 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.565009 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.565431 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.566640 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.567993 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.568415 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.568510 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.568865 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.568973 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.569040 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.569167 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.569228 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 19:43:20.576946 5346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:20.575054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.14.16.124:38583)
0807 19:43:20.575308 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.575317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.575542 (+ 225us) server_negotiation.cc:407] Connection header received
0807 19:43:20.575750 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.575755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.575841 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.575960 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.576729 (+ 769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36136: BlockingRecv error: recv got EOF from 127.0.0.1:36136 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 19:43:20.636070 5318 catalog_manager.cc:1259] Loaded cluster ID: a63a1d4a9a1845d98224bae088b8e3ba
I20260807 19:43:20.636171 5318 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330: loading cluster ID for follower catalog manager: success
I20260807 19:43:20.638520 5318 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330: acquiring CA information for follower catalog manager: success
I20260807 19:43:20.640206 5318 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1414060244dc499aa637127e18ba2330: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:43:20.713225 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.708476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35300 (local address 127.14.16.125:35117)
0807 19:43:20.708678 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.708683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.709009 (+ 326us) server_negotiation.cc:407] Connection header received
0807 19:43:20.709168 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.709173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.709248 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.709339 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.709720 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.710395 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.711761 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.712171 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.712263 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.712617 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.712729 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.712793 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.712961 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.713025 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:43:20.724849 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.719117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36146 (local address 127.14.16.124:38583)
0807 19:43:20.719372 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.719379 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.719608 (+ 229us) server_negotiation.cc:407] Connection header received
0807 19:43:20.719813 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.719819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.719984 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.720106 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.720516 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.721267 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.722599 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.723031 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.723172 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.723722 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.724252 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.724430 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.724480 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.724561 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:43:20.739723 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.730927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35310 (local address 127.14.16.125:35117)
0807 19:43:20.731283 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.731319 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.732352 (+ 1033us) server_negotiation.cc:407] Connection header received
0807 19:43:20.732579 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.732588 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.732710 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.732815 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.733487 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.734962 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.737743 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.738275 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.738393 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.738942 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.739048 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.739147 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.739368 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.739439 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260807 19:43:20.744177 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.731096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36158 (local address 127.14.16.124:38583)
0807 19:43:20.731277 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.731305 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.737877 (+ 6572us) server_negotiation.cc:407] Connection header received
0807 19:43:20.738291 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.738298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.738390 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.738473 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.739314 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.740375 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.742417 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.742916 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.743033 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.743448 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.743591 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.743658 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.743850 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.743923 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 19:43:20.745025 5484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.731158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47732 (local address 127.14.16.126:38815)
0807 19:43:20.738078 (+ 6920us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.738084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.738104 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:43:20.738222 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.738227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.738489 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.738672 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.739674 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.740686 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.742757 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.743376 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.743500 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.743954 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.744415 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.744500 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.744658 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.744725 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6793,"thread_start_us":6671,"threads_started":1}
I20260807 19:43:20.749734 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 5076
I20260807 19:43:20.751273 5140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:20.865012 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 5076
I20260807 19:43:20.890036 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 5152
I20260807 19:43:20.891211 5216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:21.003871 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 5152
I20260807 19:43:21.025590 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 5229
I20260807 19:43:21.026856 5293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:21.136971 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 5229
2026-08-07T19:43:21Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4609 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:21 dist-test-slave-07c2 krb5kdc[5491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:21 dist-test-slave-07c2 krb5kdc[5491](info): set up 2 sockets
Aug 07 19:43:21 dist-test-slave-07c2 krb5kdc[5491](info): commencing operation
krb5kdc: starting...
W20260807 19:43:22.318799 14401 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.001s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/test-user.keytab.
Aug 07 19:43:22 dist-test-slave-07c2 krb5kdc[5491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
Entry for principal kudu/127.14.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab.
I20260807 19:43:22.658371 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:46875
--webserver_interface=127.14.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:46875,127.14.16.125:40989,127.14.16.124:46859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:22.999073 5504 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:43:22.999459 5504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:22.999589 5504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:22.999722 5504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:23.007154 5504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:23.007249 5504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:23.007372 5504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:23.007460 5504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:23.007563 5504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:23.017637 5504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:46875,127.14.16.125:40989,127.14.16.124:46859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.126.keytab
--principal=kudu/127.14.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.126:46875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:23.019021 5504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:23.020433 5504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:23.032737 5510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:23.032759 5514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:23.032684 5511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:23.036839 5504 server_base.cc:1034] running on GCE node
I20260807 19:43:23.037725 5504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:23.039268 5504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:23.040580 5504 hybrid_clock.cc:648] HybridClock initialized: now 1786131803040523 us; error 40 us; skew 500 ppm
I20260807 19:43:23.040968 5504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:23 dist-test-slave-07c2 krb5kdc[5491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131803, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:23.044394 5504 init.cc:377] Logged in from keytab as kudu/127.14.16.126@KRBTEST.COM (short username kudu)
I20260807 19:43:23.046051 5504 webserver.cc:460] Webserver started at http://127.14.16.126:39687/ using document root <none> and password file <none>
I20260807 19:43:23.046803 5504 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:23.046917 5504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:23.047257 5504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:23.050405 5504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/data/instance:
uuid: "650e950881934e3fbcaba39c53d69aab"
format_stamp: "Formatted at 2026-08-07 19:43:23 on dist-test-slave-07c2"
server_key: "b6145cff97cdc8eefb23c8c3bc6e2c48"
server_key_iv: "2dd705d69ff7e6cd0f6aaac252c13ba7"
server_key_version: "encryptionkey@0"
I20260807 19:43:23.051225 5504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/wal/instance:
uuid: "650e950881934e3fbcaba39c53d69aab"
format_stamp: "Formatted at 2026-08-07 19:43:23 on dist-test-slave-07c2"
server_key: "b6145cff97cdc8eefb23c8c3bc6e2c48"
server_key_iv: "2dd705d69ff7e6cd0f6aaac252c13ba7"
server_key_version: "encryptionkey@0"
I20260807 19:43:23.056969 5504 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:43:23.060696 5521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:23.062114 5504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:43:23.062335 5504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/wal
uuid: "650e950881934e3fbcaba39c53d69aab"
format_stamp: "Formatted at 2026-08-07 19:43:23 on dist-test-slave-07c2"
server_key: "b6145cff97cdc8eefb23c8c3bc6e2c48"
server_key_iv: "2dd705d69ff7e6cd0f6aaac252c13ba7"
server_key_version: "encryptionkey@0"
I20260807 19:43:23.062592 5504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:23.087517 5504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:23.092458 5504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:23.092760 5504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:23.112401 5504 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.126:46875
I20260807 19:43:23.112424 5574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.126:46875 every 8 connection(s)
I20260807 19:43:23.113834 5504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:23.117744 5575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:23.121295 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 5504
I20260807 19:43:23.121482 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-0/wal/instance
I20260807 19:43:23.122035 14401 external_mini_cluster.cc:1428] Setting key 9c3e76d5bde7e2c4d109e2e996440662
I20260807 19:43:23.126725 5575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:43:23 dist-test-slave-07c2 krb5kdc[5491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131803, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:43:23.150619 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.130478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:46875 (local address 127.0.0.1:40652)
0807 19:43:23.131365 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.131413 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:23.133453 (+ 2040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:23.134370 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:23.134375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:23.134937 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:23.135275 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:23.135358 (+ 83us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.137677 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.137682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:23.138736 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:23.138748 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.139712 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.139719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:23.139929 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:23.140610 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:23.140633 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:23.142802 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:23.146163 (+ 3361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:23.146176 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:23.146187 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:23.146581 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:23.147003 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:23.147011 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:23.147014 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:23.147178 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:23.147976 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:23.147989 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:23.148214 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 19:43:23.148474 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 19:43:23.149195 (+ 721us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":210,"thread_start_us":387,"threads_started":1}
I20260807 19:43:23.150637 5578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.131802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40652 (local address 127.14.16.126:46875)
0807 19:43:23.132506 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:23.132516 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:23.132568 (+ 52us) server_negotiation.cc:407] Connection header received
0807 19:43:23.133820 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:23.133830 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:23.134127 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:23.134343 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:23.135556 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.137517 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.138875 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.139479 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.139678 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:23.143029 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:23.143037 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:23.143049 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:23.143077 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:23.146033 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:23.146737 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:23.146744 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:23.146746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:23.146888 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:23.147322 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:23.147329 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:23.147341 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:23.147832 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:23.148058 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:23.148611 (+ 553us) server_negotiation.cc:299] Negotiation successful
0807 19:43:23.149075 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":473,"thread_start_us":316,"threads_started":1}
I20260807 19:43:23.154052 5575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:43:23.155273 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.154886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59532)
0807 19:43:23.155049 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.155154 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 19:43:23.155627 5525 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.14.16.125@KRBTEST.COM; defaulting to no policy
I20260807 19:43:23.157321 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.156860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59542)
0807 19:43:23.157088 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.157177 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 19:43:23.157601 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:40989: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.125.keytab.
Entry for principal kudu/127.14.16.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.125.keytab.
I20260807 19:43:23.196708 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.125:40989
--webserver_interface=127.14.16.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.125.keytab
--principal=kudu/127.14.16.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:46875,127.14.16.125:40989,127.14.16.124:46859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:43:23.206944 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } attempt: 1
I20260807 19:43:23.208210 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.207757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59554)
0807 19:43:23.207959 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.208073 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 19:43:23.209326 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.208948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59560)
0807 19:43:23.209116 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.209187 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 19:43:23.209519 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:40989: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
I20260807 19:43:23.277801 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } attempt: 2
I20260807 19:43:23.279119 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.278529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59574)
0807 19:43:23.278818 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.278940 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 19:43:23.280246 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.279819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59586)
0807 19:43:23.279984 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.280089 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 19:43:23.280529 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:40989: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
I20260807 19:43:23.371838 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } attempt: 3
I20260807 19:43:23.373044 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.372606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59596)
0807 19:43:23.372814 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.372908 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 19:43:23.374138 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.373770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59602)
0807 19:43:23.373911 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.373983 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 19:43:23.374375 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:40989: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
I20260807 19:43:23.517623 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } attempt: 4
I20260807 19:43:23.518869 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.518301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59604)
0807 19:43:23.518626 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.518731 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260807 19:43:23.519963 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.519600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59620)
0807 19:43:23.519758 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.519843 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 19:43:23.520159 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.125:40989: Network error: Client connection negotiation failed: client connection to 127.14.16.125:40989: connect: Connection refused (error 111)
W20260807 19:43:23.544641 5582 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:43:23.545034 5582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:23.545166 5582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:23.545312 5582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:23.552860 5582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:23.552999 5582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:23.553123 5582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:23.553211 5582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:23.553323 5582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:23.563577 5582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:46875,127.14.16.125:40989,127.14.16.124:46859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.125.keytab
--principal=kudu/127.14.16.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.125:40989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.16.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:23.564953 5582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:23.566349 5582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:23.579191 5588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:23.579263 5587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:23.579599 5590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:23.580047 5582 server_base.cc:1034] running on GCE node
I20260807 19:43:23.581185 5582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:23.583122 5582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:23.584414 5582 hybrid_clock.cc:648] HybridClock initialized: now 1786131803584332 us; error 36 us; skew 500 ppm
I20260807 19:43:23.584776 5582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:23 dist-test-slave-07c2 krb5kdc[5491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131803, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:23.587868 5582 init.cc:377] Logged in from keytab as kudu/127.14.16.125@KRBTEST.COM (short username kudu)
I20260807 19:43:23.589443 5582 webserver.cc:460] Webserver started at http://127.14.16.125:42611/ using document root <none> and password file <none>
I20260807 19:43:23.590153 5582 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:23.590269 5582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:23.590613 5582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:23.594154 5582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/data/instance:
uuid: "50c85b428b10450e994e9021b54661cc"
format_stamp: "Formatted at 2026-08-07 19:43:23 on dist-test-slave-07c2"
server_key: "454323ab951c215afd6b6796f81ada5b"
server_key_iv: "b887d3d773693abe88595d94fc3bd466"
server_key_version: "encryptionkey@0"
I20260807 19:43:23.594962 5582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/wal/instance:
uuid: "50c85b428b10450e994e9021b54661cc"
format_stamp: "Formatted at 2026-08-07 19:43:23 on dist-test-slave-07c2"
server_key: "454323ab951c215afd6b6796f81ada5b"
server_key_iv: "b887d3d773693abe88595d94fc3bd466"
server_key_version: "encryptionkey@0"
I20260807 19:43:23.600165 5582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:43:23.603544 5598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:23.604779 5582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:43:23.604964 5582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/wal
uuid: "50c85b428b10450e994e9021b54661cc"
format_stamp: "Formatted at 2026-08-07 19:43:23 on dist-test-slave-07c2"
server_key: "454323ab951c215afd6b6796f81ada5b"
server_key_iv: "b887d3d773693abe88595d94fc3bd466"
server_key_version: "encryptionkey@0"
I20260807 19:43:23.605176 5582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:23.630826 5582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:23.634720 5582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:23.634987 5582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:23.653640 5582 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.125:40989
I20260807 19:43:23.653658 5651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.125:40989 every 8 connection(s)
I20260807 19:43:23.654982 5582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/data/info.pb
I20260807 19:43:23.658555 5652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:23.661098 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 5582
I20260807 19:43:23.661329 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-1/wal/instance
I20260807 19:43:23.661775 14401 external_mini_cluster.cc:1428] Setting key 6f690981bf360b70d7414dbcd230f071
I20260807 19:43:23.665019 5652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:43:23 dist-test-slave-07c2 krb5kdc[5491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131803, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.125@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:43:23.684757 5654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.667673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40666 (local address 127.14.16.126:46875)
0807 19:43:23.668184 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:23.668189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:23.669124 (+ 935us) server_negotiation.cc:407] Connection header received
0807 19:43:23.670917 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:23.670923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:23.671015 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:23.671135 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:23.672640 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.673404 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.674879 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.675345 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.675479 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:23.678838 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:23.678843 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:23.678846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:23.678882 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:23.681671 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:23.682398 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:23.682406 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:23.682412 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:23.682537 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:23.682948 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:23.682954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:23.682957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:23.683241 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:23.683451 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:23.684147 (+ 696us) server_negotiation.cc:299] Negotiation successful
0807 19:43:23.684369 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":279,"threads_started":1}
I20260807 19:43:23.685768 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.668223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:46875 (local address 127.0.0.1:40666)
0807 19:43:23.668975 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.669011 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:23.670655 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:23.671409 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:23.671417 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:23.671909 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:23.672424 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:23.672440 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.673574 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.673577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:23.674735 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:23.674747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.675564 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.675571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:23.675799 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:23.676509 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:23.676530 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:23.678624 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:23.681821 (+ 3197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:23.681829 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:23.681843 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:23.682279 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:23.682649 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:23.682654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:23.682657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:23.682832 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:23.683434 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:23.683441 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:23.683800 (+ 359us) client_negotiation.cc:769] Sending connection context
0807 19:43:23.684069 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 19:43:23.684566 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":320,"threads_started":1}
I20260807 19:43:23.687536 5652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.14.16.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.16.124@KRBTEST.COM" created.
Aug 07 19:43:23 dist-test-slave-07c2 krb5kdc[5491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131803, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.125@KRBTEST.COM for kudu/127.14.16.125@KRBTEST.COM
I20260807 19:43:23.703346 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.688561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59636 (local address 127.14.16.125:40989)
0807 19:43:23.689210 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:23.689216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:23.689239 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:43:23.689280 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:23.689286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:23.689443 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:23.689552 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:23.690607 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.691843 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.693314 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.693878 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.694010 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:23.697235 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:23.697247 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:23.697260 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:23.697292 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:23.700434 (+ 3142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:23.701243 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:23.701249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:23.701251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:23.701364 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:23.701779 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:23.701783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:23.701784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:23.702102 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:23.702264 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:23.702671 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 19:43:23.702994 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":502,"thread_start_us":368,"threads_started":1}
I20260807 19:43:23.703423 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.688350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59636)
0807 19:43:23.688578 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.688603 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:23.688715 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:23.689615 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:23.689621 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:23.690181 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:23.690415 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:23.690432 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.692012 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.692016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:23.693175 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:23.693186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.694030 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.694035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:23.694240 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:23.694827 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:23.694843 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:23.697012 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:23.700567 (+ 3555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:23.700575 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:23.700581 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:23.701111 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:23.701466 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:23.701473 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:23.701479 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:23.701676 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:23.702227 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:23.702234 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:23.702396 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 19:43:23.702554 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 19:43:23.702879 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 19:43:23.706199 5652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:43:23.707517 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.706960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60228)
0807 19:43:23.707119 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.707266 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 19:43:23.707815 5600 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
I20260807 19:43:23.709493 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.709069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60240)
0807 19:43:23.709254 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.709384 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 19:43:23.709704 5652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46859: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.16.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.124.keytab.
Entry for principal kudu/127.14.16.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.124.keytab.
I20260807 19:43:23.734654 14401 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
/tmp/dist-test-taskBvS7k8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.124:46859
--webserver_interface=127.14.16.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.124.keytab
--principal=kudu/127.14.16.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.16.84:42205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.16.126:46875,127.14.16.125:40989,127.14.16.124:46859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:43:23.758997 5652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } attempt: 1
I20260807 19:43:23.760211 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.759717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60256)
0807 19:43:23.759924 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.760050 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 19:43:23.761279 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.760931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60272)
0807 19:43:23.761103 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.761175 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 19:43:23.761484 5652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46859: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
I20260807 19:43:23.819418 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } attempt: 5
Aug 07 19:43:23 dist-test-slave-07c2 krb5kdc[5491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131803, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for kudu/127.14.16.125@KRBTEST.COM
I20260807 19:43:23.829696 5652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } attempt: 2
I20260807 19:43:23.831099 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.830297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60274)
0807 19:43:23.830785 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.830896 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331}
I20260807 19:43:23.832401 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.831968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60288)
0807 19:43:23.832152 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.832235 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 19:43:23.832623 5652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46859: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
I20260807 19:43:23.834800 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.820307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.14.16.125:40989)
0807 19:43:23.820471 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:23.820475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:23.820565 (+ 90us) server_negotiation.cc:407] Connection header received
0807 19:43:23.820816 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:23.820820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:23.820887 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:23.821006 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:23.822051 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.822739 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.824485 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.825109 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.825224 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:23.828816 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:23.828821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:23.828824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:23.828857 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:23.832206 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:23.833009 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:23.833013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:23.833015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:23.833127 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:23.833512 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:23.833515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:23.833517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:23.833790 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:23.833897 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:23.834332 (+ 435us) server_negotiation.cc:299] Negotiation successful
0807 19:43:23.834482 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 19:43:23.834899 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.820136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59652)
0807 19:43:23.820443 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.820469 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:23.820656 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:23.821080 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:23.821087 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:23.821649 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:23.821890 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:23.821907 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.822878 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.822883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:23.824295 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:23.824311 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:23.825269 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:23.825275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:23.825527 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:23.826187 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:23.826210 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:23.828613 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:23.832352 (+ 3739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:23.832360 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:23.832365 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:23.832896 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:23.833236 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:23.833241 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:23.833244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:23.833409 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:23.833907 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:23.833912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:23.834082 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 19:43:23.834244 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 19:43:23.834552 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 19:43:23.836369 5575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:43:23.837455 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.836992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60302)
0807 19:43:23.837170 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.837278 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 19:43:23.838454 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.838070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60310)
0807 19:43:23.838233 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.838337 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 19:43:23.838667 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46859: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
I20260807 19:43:23.889928 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } attempt: 1
I20260807 19:43:23.891270 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.890709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60322)
0807 19:43:23.890943 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.891057 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 19:43:23.892356 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.891978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60332)
0807 19:43:23.892132 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.892217 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 19:43:23.892589 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46859: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
I20260807 19:43:23.923841 5652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } attempt: 3
I20260807 19:43:23.924991 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.924522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60334)
0807 19:43:23.924744 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.924838 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 19:43:23.926198 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.925737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60338)
0807 19:43:23.925971 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.926065 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260807 19:43:23.926445 5652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46859: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
I20260807 19:43:23.960844 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } attempt: 2
I20260807 19:43:23.962253 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.961638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60344)
0807 19:43:23.961898 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.962026 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 19:43:23.963263 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:23.962903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60354)
0807 19:43:23.963051 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:23.963142 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 19:43:23.963510 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46859: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
I20260807 19:43:24.069680 5652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } attempt: 4
I20260807 19:43:24.069762 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } attempt: 3
I20260807 19:43:24.071034 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.070386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60364)
0807 19:43:24.070656 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:24.070799 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 19:43:24.071084 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.070385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60362)
0807 19:43:24.070744 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:24.070867 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
I20260807 19:43:24.072288 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.071846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60392)
0807 19:43:24.072074 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:24.072161 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260807 19:43:24.072516 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.071916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60404)
0807 19:43:24.072166 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:24.072244 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"threads_started":1}
W20260807 19:43:24.072496 5652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46859: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
W20260807 19:43:24.073081 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.16.124:46859: Network error: Client connection negotiation failed: client connection to 127.14.16.124:46859: connect: Connection refused (error 111)
W20260807 19:43:24.079270 5660 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:43:24.079668 5660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:24.079774 5660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:24.079907 5660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:24.087158 5660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:24.087244 5660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:24.087384 5660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:24.087473 5660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:24.087563 5660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:24.097694 5660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.16.84:42205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.16.126:46875,127.14.16.125:40989,127.14.16.124:46859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/krb5kdc/kudu_127.14.16.124.keytab
--principal=kudu/127.14.16.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.16.124:46859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.16.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:43:24.099006 5660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:24.100399 5660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:24.112545 5666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:24.112556 5667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:24.113219 5669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:24.116851 5660 server_base.cc:1034] running on GCE node
I20260807 19:43:24.117654 5660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:24.119063 5660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:24.120319 5660 hybrid_clock.cc:648] HybridClock initialized: now 1786131804120178 us; error 116 us; skew 500 ppm
I20260807 19:43:24.120671 5660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:24 dist-test-slave-07c2 krb5kdc[5491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131804, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:24.123796 5660 init.cc:377] Logged in from keytab as kudu/127.14.16.124@KRBTEST.COM (short username kudu)
I20260807 19:43:24.125319 5660 webserver.cc:460] Webserver started at http://127.14.16.124:37273/ using document root <none> and password file <none>
I20260807 19:43:24.126019 5660 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:24.126159 5660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:24.126453 5660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:24.129375 5660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/data/instance:
uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13"
format_stamp: "Formatted at 2026-08-07 19:43:24 on dist-test-slave-07c2"
server_key: "b397d520436e0c110be4414bf52b5602"
server_key_iv: "daf734d9425dc14519149b0d54484a8a"
server_key_version: "encryptionkey@0"
I20260807 19:43:24.130115 5660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/wal/instance:
uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13"
format_stamp: "Formatted at 2026-08-07 19:43:24 on dist-test-slave-07c2"
server_key: "b397d520436e0c110be4414bf52b5602"
server_key_iv: "daf734d9425dc14519149b0d54484a8a"
server_key_version: "encryptionkey@0"
I20260807 19:43:24.135151 5660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:43:24.138566 5676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:24.140074 5660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:43:24.140226 5660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/data,/tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/wal
uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13"
format_stamp: "Formatted at 2026-08-07 19:43:24 on dist-test-slave-07c2"
server_key: "b397d520436e0c110be4414bf52b5602"
server_key_iv: "daf734d9425dc14519149b0d54484a8a"
server_key_version: "encryptionkey@0"
I20260807 19:43:24.140408 5660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:24.161226 5660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:24.165230 5660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:24.165448 5660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:24.185036 5660 rpc_server.cc:307] RPC server started. Bound to: 127.14.16.124:46859
I20260807 19:43:24.185181 5729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.16.124:46859 every 8 connection(s)
I20260807 19:43:24.186445 5660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/data/info.pb
I20260807 19:43:24.187615 14401 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu as pid 5660
I20260807 19:43:24.187821 14401 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBvS7k8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131435615921-14401-0/minicluster-data/master-2/wal/instance
I20260807 19:43:24.188282 14401 external_mini_cluster.cc:1428] Setting key 99bdff0a6944263b21ce6b61df017c28
I20260807 19:43:24.190883 5730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:24.199322 5730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:43:24 dist-test-slave-07c2 krb5kdc[5491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.124@KRBTEST.COM
Aug 07 19:43:24 dist-test-slave-07c2 krb5kdc[5491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131804, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.124@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
I20260807 19:43:24.213393 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.190200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60416 (local address 127.14.16.124:46859)
0807 19:43:24.191042 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.191058 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.191108 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:43:24.192840 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.192860 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.193251 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.193603 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.195555 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.196915 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.198719 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.199454 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.199603 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.204740 (+ 5137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:24.204753 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:24.204768 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:24.204811 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:24.208877 (+ 4066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.209545 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.209553 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.209562 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.209812 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.210259 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.210266 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.210271 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.211029 (+ 758us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:24.211341 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:24.211810 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 19:43:24.212217 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":378,"threads_started":1}
I20260807 19:43:24.219448 5735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.202564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.126:46875 (local address 127.0.0.1:40676)
0807 19:43:24.203550 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:24.203600 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:24.203766 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:24.204186 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:24.204191 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:24.205031 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:24.205456 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:24.205473 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.206693 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.206698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:24.208587 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:24.208603 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.209451 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.209458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:24.209636 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.210395 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:24.210416 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:24.212963 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:24.216664 (+ 3701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:24.216676 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:24.216682 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:24.217210 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:24.217618 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:24.217623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:24.217626 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:24.217813 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:24.218336 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:24.218344 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:24.218561 (+ 217us) client_negotiation.cc:769] Sending connection context
0807 19:43:24.218771 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 19:43:24.219078 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":571,"threads_started":1}
I20260807 19:43:24.219591 5734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.202554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40676 (local address 127.14.16.126:46875)
0807 19:43:24.203165 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.203172 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.203704 (+ 532us) server_negotiation.cc:407] Connection header received
0807 19:43:24.203904 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.203912 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.204007 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.204135 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.205639 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.206475 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.208785 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.209268 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.209417 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.213156 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:24.213162 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:24.213167 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:24.213215 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:24.216496 (+ 3281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.217330 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.217336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.217339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.217510 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.217934 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.217937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.217940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.218200 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:24.218319 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:24.218869 (+ 550us) server_negotiation.cc:299] Negotiation successful
0807 19:43:24.219123 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":277,"threads_started":1}
I20260807 19:43:24.220813 5730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:43:24 dist-test-slave-07c2 krb5kdc[5491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.126@KRBTEST.COM
Aug 07 19:43:24 dist-test-slave-07c2 krb5kdc[5491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131804, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.124@KRBTEST.COM for kudu/127.14.16.125@KRBTEST.COM
I20260807 19:43:24.233166 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.216611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.14.16.126:46875)
0807 19:43:24.217239 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.217244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.217285 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:43:24.217332 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.217337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.217407 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.217522 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.219021 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.220482 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.222044 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.222694 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.222828 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.227212 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:24.227220 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:24.227225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:24.227263 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:24.230172 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.230916 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.230919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.230922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.231053 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.231579 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.231586 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.231589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.231856 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:24.232152 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:24.232469 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 19:43:24.232688 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":315,"threads_started":1}
I20260807 19:43:24.235042 5735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.221461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.125:40989 (local address 127.0.0.1:59666)
0807 19:43:24.221661 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:24.221680 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:24.221780 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:24.222199 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:24.222204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:24.222659 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:24.222853 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:24.222864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.224221 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.224224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:24.225075 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:24.225085 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.226020 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.226026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:24.226236 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.226794 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:24.226811 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:24.229184 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:24.232484 (+ 3300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:24.232490 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:24.232495 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:24.232892 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:24.233364 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:24.233369 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:24.233373 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:24.233533 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:24.234134 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:24.234139 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:24.234273 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 19:43:24.234433 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 19:43:24.234712 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 19:43:24.235419 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.221671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59666 (local address 127.14.16.125:40989)
0807 19:43:24.221887 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.221893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.221913 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:43:24.221974 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.221984 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.222064 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.222150 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.223011 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.224075 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.225222 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.225859 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.225950 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.229384 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:24.229388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:24.229391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:24.229424 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:24.232349 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.233028 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.233033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.233037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.233216 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.233647 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.233652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.233655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.233997 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:24.234143 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:24.234530 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 19:43:24.234747 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:43:24.236377 5730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:43:24 dist-test-slave-07c2 krb5kdc[5491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131804, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.124@KRBTEST.COM for kudu/127.14.16.124@KRBTEST.COM
Aug 07 19:43:24 dist-test-slave-07c2 krb5kdc[5491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.16.125@KRBTEST.COM
I20260807 19:43:24.250320 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } attempt: 4
I20260807 19:43:24.251101 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.237294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60430 (local address 127.14.16.124:46859)
0807 19:43:24.237452 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.237464 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.237536 (+ 72us) server_negotiation.cc:407] Connection header received
0807 19:43:24.237725 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.237729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.237811 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.237910 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.238538 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.239330 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.240778 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.241326 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.241451 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.245165 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:24.245169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:24.245172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:24.245204 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:24.248462 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.249078 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.249084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.249087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.249240 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.249681 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.249684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.249686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.249979 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:24.250123 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:24.250471 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 19:43:24.250706 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:43:24.252067 5735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.237294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60430)
0807 19:43:24.237466 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:24.237485 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:24.237594 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:24.237940 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:24.237943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:24.238276 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:24.238407 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:24.238415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.239462 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.239467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:24.240651 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:24.240663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.241505 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.241511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:24.241761 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.242274 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:24.242300 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:24.245010 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:24.248602 (+ 3592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:24.248606 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:24.248609 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:24.248960 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:24.249376 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:24.249380 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:24.249383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:24.249567 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:24.250096 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:24.250100 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:24.250224 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 19:43:24.250375 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 19:43:24.251685 (+ 1310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":925}
I20260807 19:43:24.253211 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.235252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59668 (local address 127.14.16.125:40989)
0807 19:43:24.236113 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.236122 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.236142 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:43:24.236194 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.236201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.236291 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.236414 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.237826 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.238700 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.240313 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.240757 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.240877 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.245987 (+ 5110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:24.245993 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:24.245998 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:24.246037 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:24.249255 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.250026 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.250033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.250038 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.250211 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.250756 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.250760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.250762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.251112 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:24.251459 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:24.252593 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0807 19:43:24.252837 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":1400,"threads_started":1}
I20260807 19:43:24.258445 5730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13: Bootstrap starting.
I20260807 19:43:24.259156 14401 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 07 19:43:24 dist-test-slave-07c2 krb5kdc[5491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131803, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.126@KRBTEST.COM for kudu/127.14.16.124@KRBTEST.COM
I20260807 19:43:24.262243 5730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:24.263463 5730 log.cc:824] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:24.262032 5549 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "650e950881934e3fbcaba39c53d69aab"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40686
I20260807 19:43:24.266332 5730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13: No bootstrap required, opened a new log
I20260807 19:43:24.268561 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.250995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60436)
0807 19:43:24.251279 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:24.251321 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:24.251451 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:24.251838 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:24.251844 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:24.252509 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:24.252832 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:24.252845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.254508 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.254517 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:24.255999 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:24.256019 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.257026 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.257034 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:24.257234 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.257939 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:24.257964 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:24.261288 (+ 3324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:24.265488 (+ 4200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:24.265494 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:24.265498 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:24.266153 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:24.266627 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:24.266632 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:24.266636 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:24.266778 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:24.267518 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:24.267522 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:24.267698 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 19:43:24.267837 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 19:43:24.268124 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260807 19:43:24.268579 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.251052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60436 (local address 127.14.16.124:46859)
0807 19:43:24.251409 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.251414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.251431 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:43:24.251549 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.251554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.251647 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.252982 (+ 1335us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.253118 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.254242 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.256153 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.256839 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.256974 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.261482 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:24.261488 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:24.261493 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:24.261527 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:24.265321 (+ 3794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.266346 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.266354 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.266357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.266520 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.266898 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.266903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.266906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.267393 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:24.267572 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:24.267964 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 19:43:24.268168 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":684,"threads_started":1}
I20260807 19:43:24.272816 5575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab: Bootstrap starting.
I20260807 19:43:24.275806 5730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } }
I20260807 19:43:24.276149 5730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447b2f4ba6c34e8e96f3edd1f28c4f13, State: Initialized, Role: FOLLOWER
I20260807 19:43:24.276607 5730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } }
I20260807 19:43:24.276731 5575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:24.277554 5740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } } }
I20260807 19:43:24.277935 5740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:24.277987 5575 log.cc:824] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:24.278800 5730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:24.281260 5575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab: No bootstrap required, opened a new log
I20260807 19:43:24.287956 5575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } }
I20260807 19:43:24.288342 5575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 650e950881934e3fbcaba39c53d69aab, State: Initialized, Role: FOLLOWER
I20260807 19:43:24.288861 5575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } }
W20260807 19:43:24.290059 5752 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:43:24.290230 5752 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:43:24.295884 5754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } } }
I20260807 19:43:24.296409 5754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:24.297168 5575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:24.297310 5549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:43:24.297506 5549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:24.287508 5660 master_runner.cc:186] Error getting master registration for 127.14.16.126:46875: OK, instance_id { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" instance_seqno: 1786131803106818 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:43:24.301178 5549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } }
I20260807 19:43:24.301467 5660 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:43:24.303812 5705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e950881934e3fbcaba39c53d69aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13"
I20260807 19:43:24.304287 5705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:24.306972 5549 leader_election.cc:288] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [CANDIDATE]: Term 1 election: Requested vote from peers 50c85b428b10450e994e9021b54661cc (127.14.16.125:40989), 447b2f4ba6c34e8e96f3edd1f28c4f13 (127.14.16.124:46859)
I20260807 19:43:24.308533 5705 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 650e950881934e3fbcaba39c53d69aab in term 1.
I20260807 19:43:24.309293 5523 leader_election.cc:302] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 447b2f4ba6c34e8e96f3edd1f28c4f13, 650e950881934e3fbcaba39c53d69aab; no voters:
I20260807 19:43:24.309953 5754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:24.314276 5754 raft_consensus.cc:686] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [term 1 LEADER]: Becoming Leader. State: Replica: 650e950881934e3fbcaba39c53d69aab, State: Running, Role: LEADER
I20260807 19:43:24.314936 5754 consensus_queue.cc:237] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } }
I20260807 19:43:24.322872 5758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 650e950881934e3fbcaba39c53d69aab. Latest consensus state: current_term: 1 leader_uuid: "650e950881934e3fbcaba39c53d69aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } } }
I20260807 19:43:24.323401 5758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:24.329211 5763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:24.342478 5763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 19:43:24.345671 5527 tablet.cc:2369] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:43:24.366582 5705 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [term 1 FOLLOWER]: Refusing update from remote peer 650e950881934e3fbcaba39c53d69aab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:24.367895 5758 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [LEADER]: Connected to new peer: Peer: permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:24.371678 5652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } attempt: 5
I20260807 19:43:24.383278 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.338923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.14.16.125:40989)
0807 19:43:24.339105 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.339111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.348442 (+ 9331us) server_negotiation.cc:407] Connection header received
0807 19:43:24.348936 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.348944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.349033 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.349110 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.359433 (+ 10323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.360894 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.363330 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.363961 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.364076 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.367710 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:24.367717 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:24.367721 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:24.367765 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:24.371419 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.380339 (+ 8920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.380348 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.380355 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.380590 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.381183 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.381190 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.381196 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.381557 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:24.381817 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:24.382498 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 19:43:24.382847 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 19:43:24 dist-test-slave-07c2 krb5kdc[5491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131803, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.16.125@KRBTEST.COM for kudu/127.14.16.124@KRBTEST.COM
I20260807 19:43:24.393939 5740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 650e950881934e3fbcaba39c53d69aab. Latest consensus state: current_term: 1 leader_uuid: "650e950881934e3fbcaba39c53d69aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } } }
I20260807 19:43:24.394265 5740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:24.396986 5770 mvcc.cc:205] Tried to move back new op lower bound from 7315995870671572992 to 7315995870480306176. Current Snapshot: MvccSnapshot[applied={T|T < 7315995870671572992}]
I20260807 19:43:24.400408 5774 mvcc.cc:205] Tried to move back new op lower bound from 7315995870671572992 to 7315995870480306176. Current Snapshot: MvccSnapshot[applied={T|T < 7315995870671572992}]
I20260807 19:43:24.408259 5754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "650e950881934e3fbcaba39c53d69aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } } }
I20260807 19:43:24.408663 5754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:24.410096 5763 catalog_manager.cc:1347] Generated new cluster ID: 325875ae06ab42839b79a696c8f3eac9
I20260807 19:43:24.410209 5763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:24.425407 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.372215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.16.124:46859 (local address 127.0.0.1:60462)
0807 19:43:24.372579 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:24.372599 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:24.372713 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:24.373476 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:24.373484 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:24.374111 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:24.374371 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:24.374384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.377701 (+ 3317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.377709 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:24.380471 (+ 2762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:24.380494 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.382449 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.382458 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:24.383210 (+ 752us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.384977 (+ 1767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:24.385042 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:24.394016 (+ 8974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:24.404944 (+ 10928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:24.404953 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:24.404960 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:24.406378 (+ 1418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:24.410759 (+ 4381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:24.410793 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:24.410797 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:24.411094 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:24.416974 (+ 5880us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:24.416980 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:24.417314 (+ 334us) client_negotiation.cc:769] Sending connection context
0807 19:43:24.418623 (+ 1309us) client_negotiation.cc:240] Negotiation successful
0807 19:43:24.419811 (+ 1188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246}
I20260807 19:43:24.427562 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.372550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60462 (local address 127.14.16.124:46859)
0807 19:43:24.372739 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.372746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.372774 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:43:24.372829 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.372837 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.372939 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.373067 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.375822 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.377465 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.380728 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.381927 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.382077 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.394399 (+ 12322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:24.394409 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:24.394416 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:24.394461 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:24.404074 (+ 9613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.407170 (+ 3096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.407176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.407397 (+ 221us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.410054 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.411834 (+ 1780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.411840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.411844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.415968 (+ 4124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:24.420465 (+ 4497us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:24.422263 (+ 1798us) server_negotiation.cc:299] Negotiation successful
0807 19:43:24.425759 (+ 3496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 19:43:24.427649 5782 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:43:24.433485 5652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc: Bootstrap starting.
I20260807 19:43:24.437739 5504 master_runner.cc:186] Error getting master registration for 127.14.16.125:40989: OK, instance_id { permanent_uuid: "50c85b428b10450e994e9021b54661cc" instance_seqno: 1786131803648365 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:43:24.449846 5504 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:43:24.451822 5652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:24.453294 5652 log.cc:824] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:24.457774 5740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "650e950881934e3fbcaba39c53d69aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } } }
I20260807 19:43:24.458068 5740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:24.464075 5763 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:24.465924 5763 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:24.475755 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.333776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60452 (local address 127.14.16.124:46859)
0807 19:43:24.333956 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.333962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.347610 (+ 13648us) server_negotiation.cc:407] Connection header received
0807 19:43:24.348465 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.348475 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.348628 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.348723 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.366167 (+ 17444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.368843 (+ 2676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.408635 (+ 39792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.409644 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.409775 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.465903 (+ 56128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:24.465912 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:24.465918 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:24.465959 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:24.469940 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.471723 (+ 1783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.471729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.471733 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.471912 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.473534 (+ 1622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.473540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.473543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.473921 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:24.474183 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:24.475070 (+ 887us) server_negotiation.cc:299] Negotiation successful
0807 19:43:24.475349 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 19:43:24.485015 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:24.333959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.14.16.126:46875)
0807 19:43:24.334219 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.334227 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.347458 (+ 13231us) server_negotiation.cc:407] Connection header received
0807 19:43:24.349075 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.349082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.349223 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.349326 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.413951 (+ 64625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.415084 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.459430 (+ 44346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:24.460150 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:24.460278 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:24.477067 (+ 16789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:24.477076 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:24.477084 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:24.477127 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:24.481224 (+ 4097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.482404 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.482410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.482414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.482571 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:24.483087 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:24.483092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:24.483095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:24.483520 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:24.483777 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:24.484367 (+ 590us) server_negotiation.cc:299] Negotiation successful
0807 19:43:24.484630 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:43:24.489449 5652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc: No bootstrap required, opened a new log
I20260807 19:43:24.490414 5763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab: Generated new TSK 0
I20260807 19:43:24.492491 5763 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:43:24.502918 5652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } }
I20260807 19:43:24.503520 5652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50c85b428b10450e994e9021b54661cc, State: Initialized, Role: FOLLOWER
I20260807 19:43:24.504508 5652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } }
I20260807 19:43:24.523895 5784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } } }
I20260807 19:43:24.524438 5784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:24.525491 5652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:24.525579 5626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "650e950881934e3fbcaba39c53d69aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "50c85b428b10450e994e9021b54661cc"
I20260807 19:43:24.526074 5626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:24.529685 5626 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 650e950881934e3fbcaba39c53d69aab in term 1.
I20260807 19:43:24.530788 5625 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [term 1 FOLLOWER]: Refusing update from remote peer 650e950881934e3fbcaba39c53d69aab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:24.532397 5758 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:24.588773 5784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 650e950881934e3fbcaba39c53d69aab. Latest consensus state: current_term: 1 leader_uuid: "650e950881934e3fbcaba39c53d69aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } } }
I20260807 19:43:24.589221 5784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:24.588773 5796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "650e950881934e3fbcaba39c53d69aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } } }
I20260807 19:43:24.589423 5796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:43:24.598984 5803 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260807 19:43:24.638880 14401 mini_kdc.cc:348] Time spent kdestroy: real 0.135s user 0.003s sys 0.029s
W20260807 19:43:24.658013 5733 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:24.642923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60464 (local address 127.14.16.124:46859)
0807 19:43:24.643175 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.643184 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.656251 (+ 13067us) server_negotiation.cc:407] Connection header received
0807 19:43:24.656448 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.656454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.656535 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.656633 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.657727 (+ 1094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60464: BlockingRecv error: recv got EOF from 127.0.0.1:60464 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260807 19:43:24.659180 5736 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:24.642841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40708 (local address 127.14.16.126:46875)
0807 19:43:24.643111 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.643122 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.655928 (+ 12806us) server_negotiation.cc:407] Connection header received
0807 19:43:24.656139 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.656148 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.656266 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.656391 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.658828 (+ 2437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40708: BlockingRecv error: recv got EOF from 127.0.0.1:40708 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 19:43:24.660135 5657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:24.649010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59678 (local address 127.14.16.125:40989)
0807 19:43:24.649264 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.649272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.658015 (+ 8743us) server_negotiation.cc:407] Connection header received
0807 19:43:24.658194 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.658203 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.658319 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.658424 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.659807 (+ 1383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59678: BlockingRecv error: recv got EOF from 127.0.0.1:59678 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 19:43:24.694490 5657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:24.686197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59694 (local address 127.14.16.125:40989)
0807 19:43:24.686373 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.686381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.692791 (+ 6410us) server_negotiation.cc:407] Connection header received
0807 19:43:24.693037 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.693044 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.693181 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.693300 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.694215 (+ 915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59694: BlockingRecv error: recv got EOF from 127.0.0.1:59694 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 19:43:24.701531 5657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:24.699447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.14.16.125:40989)
0807 19:43:24.699602 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.699608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.700084 (+ 476us) server_negotiation.cc:407] Connection header received
0807 19:43:24.700273 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.700279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.700378 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.700496 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.701313 (+ 817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59708: BlockingRecv error: recv got EOF from 127.0.0.1:59708 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 19:43:24.711767 5657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:24.709693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59718 (local address 127.14.16.125:40989)
0807 19:43:24.709864 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.709869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.710353 (+ 484us) server_negotiation.cc:407] Connection header received
0807 19:43:24.710521 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.710527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.710635 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.710745 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.711524 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59718: BlockingRecv error: recv got EOF from 127.0.0.1:59718 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 19:43:24.721043 5657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:24.718758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59732 (local address 127.14.16.125:40989)
0807 19:43:24.718927 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.718933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.719492 (+ 559us) server_negotiation.cc:407] Connection header received
0807 19:43:24.719681 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.719688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.719803 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.719928 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.720777 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59732: BlockingRecv error: recv got EOF from 127.0.0.1:59732 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 19:43:24.734823 5657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:24.732673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.14.16.125:40989)
0807 19:43:24.732829 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.732835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.733373 (+ 538us) server_negotiation.cc:407] Connection header received
0807 19:43:24.733549 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.733556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.733656 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.733776 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.734576 (+ 800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59746: BlockingRecv error: recv got EOF from 127.0.0.1:59746 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 19:43:24.757200 5657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:24.754731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59756 (local address 127.14.16.125:40989)
0807 19:43:24.754931 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.754937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.755498 (+ 561us) server_negotiation.cc:407] Connection header received
0807 19:43:24.755705 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.755712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.755829 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.755951 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.756892 (+ 941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59756: BlockingRecv error: recv got EOF from 127.0.0.1:59756 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 19:43:24.795554 5657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:24.792968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.14.16.125:40989)
0807 19:43:24.793178 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.793185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.793745 (+ 560us) server_negotiation.cc:407] Connection header received
0807 19:43:24.793988 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.793997 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.794141 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.794278 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.795235 (+ 957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59762: BlockingRecv error: recv got EOF from 127.0.0.1:59762 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 19:43:24.870136 5657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:24.867918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59774 (local address 127.14.16.125:40989)
0807 19:43:24.868103 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:24.868109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:24.868624 (+ 515us) server_negotiation.cc:407] Connection header received
0807 19:43:24.868805 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:24.868812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:24.868921 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:24.869040 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:24.869887 (+ 847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59774: BlockingRecv error: recv got EOF from 127.0.0.1:59774 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 19:43:25.003413 5657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:25.001357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.14.16.125:40989)
0807 19:43:25.001603 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:25.001609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:25.001909 (+ 300us) server_negotiation.cc:407] Connection header received
0807 19:43:25.002090 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:25.002095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:25.002180 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:25.002270 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:25.003122 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59788: BlockingRecv error: recv got EOF from 127.0.0.1:59788 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:43:25.062785 5758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "650e950881934e3fbcaba39c53d69aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "650e950881934e3fbcaba39c53d69aab" member_type: VOTER last_known_addr { host: "127.14.16.126" port: 46875 } } peers { permanent_uuid: "50c85b428b10450e994e9021b54661cc" member_type: VOTER last_known_addr { host: "127.14.16.125" port: 40989 } } peers { permanent_uuid: "447b2f4ba6c34e8e96f3edd1f28c4f13" member_type: VOTER last_known_addr { host: "127.14.16.124" port: 46859 } } }
I20260807 19:43:25.063169 5758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650e950881934e3fbcaba39c53d69aab [sys.catalog]: This master's current role is: LEADER
W20260807 19:43:25.265353 5657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:25.263650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59800 (local address 127.14.16.125:40989)
0807 19:43:25.263860 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:25.263864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:25.264112 (+ 248us) server_negotiation.cc:407] Connection header received
0807 19:43:25.264294 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:25.264298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:25.264374 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:25.264472 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:25.265162 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59800: BlockingRecv error: recv got EOF from 127.0.0.1:59800 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:43:25.292339 5752 catalog_manager.cc:1259] Loaded cluster ID: 325875ae06ab42839b79a696c8f3eac9
I20260807 19:43:25.292445 5752 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13: loading cluster ID for follower catalog manager: success
I20260807 19:43:25.294940 5752 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13: acquiring CA information for follower catalog manager: success
I20260807 19:43:25.296871 5752 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 447b2f4ba6c34e8e96f3edd1f28c4f13: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:43:25.603361 5803 catalog_manager.cc:1259] Loaded cluster ID: 325875ae06ab42839b79a696c8f3eac9
I20260807 19:43:25.603524 5803 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc: loading cluster ID for follower catalog manager: success
I20260807 19:43:25.606101 5803 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc: acquiring CA information for follower catalog manager: success
I20260807 19:43:25.607954 5803 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 50c85b428b10450e994e9021b54661cc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:43:25.787894 5872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:25.781550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.14.16.125:40989)
0807 19:43:25.782411 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:25.782418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:25.782437 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:43:25.782512 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:25.782516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:25.782731 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:25.782869 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:25.783345 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:25.784156 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:25.785531 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:25.786025 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:25.786170 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:25.786709 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:25.787194 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:25.787391 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:25.787473 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 19:43:25.787562 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":538,"threads_started":1}
I20260807 19:43:25.804143 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:25.794606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60474 (local address 127.14.16.124:46859)
0807 19:43:25.798111 (+ 3505us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:25.798116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:25.798132 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:43:25.798225 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:25.798228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:25.798377 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:25.798517 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:25.799093 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:25.799881 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:25.801854 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:25.802244 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:25.802339 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:25.802929 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:25.803482 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:25.803589 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:25.803748 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 19:43:25.803813 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3403,"thread_start_us":3307,"threads_started":1}
I20260807 19:43:25.822650 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:25.813525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.14.16.124:46859)
0807 19:43:25.813734 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:25.813743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:25.816563 (+ 2820us) server_negotiation.cc:407] Connection header received
0807 19:43:25.816751 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:25.816759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:25.816880 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:25.816985 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:25.817435 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:25.819092 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:25.820671 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:25.821262 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:25.821408 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:25.821916 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:25.822014 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:25.822118 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:25.822276 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 19:43:25.822355 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:43:25.826581 5872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:25.813925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.14.16.125:40989)
0807 19:43:25.814143 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:25.814151 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:25.817760 (+ 3609us) server_negotiation.cc:407] Connection header received
0807 19:43:25.817883 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:25.817889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:25.817979 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:25.818082 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:25.818823 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:25.820047 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:25.824329 (+ 4282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:25.825058 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:25.825193 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:25.825682 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:25.825810 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:25.825911 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:25.826079 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 19:43:25.826182 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:43:25.828413 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:25.813375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.14.16.126:46875)
0807 19:43:25.816845 (+ 3470us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:25.816852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:25.816873 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:43:25.817009 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:25.817014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:25.817230 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:25.817412 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:25.818489 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:25.820617 (+ 2128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:25.823158 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:25.823821 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:25.824305 (+ 484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:25.826018 (+ 1713us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:25.826672 (+ 654us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:25.826793 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:25.827848 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0807 19:43:25.828187 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3353,"thread_start_us":3246,"threads_started":1}
I20260807 19:43:25.833247 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 5504
I20260807 19:43:25.834339 5570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:25.947927 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 5504
I20260807 19:43:25.973150 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 5582
I20260807 19:43:25.974243 5646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:26.081598 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 5582
I20260807 19:43:26.105084 14401 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu pid 5660
I20260807 19:43:26.106220 5725 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:26.213945 14401 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBvS7k8/build/asan/bin/kudu with pid 5660
2026-08-07T19:43:26Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5075 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9685 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (370564 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST